half-baked thought: for programming languages, the...
# thinking-together
g
half-baked thought: for programming languages, the environment (in the lisp sense: “which variables are in scope and what are their values”) is really just a poorly constructed database we blindly query against by typing symbols and seeing what they resolve to (or if they resolve to anything). Most (all?) programming activity can be boiled down to: 1. adding a key/value to the database (eg by defining a function or variable) 2. finding a value (calling a function, referencing a variable) 3. doing some transformation with the value, usually involving 1 or 2 to find instructions for the work to be done (eg, resolving the symbol ‘+’) 4. querying a location on the OS and receiving or sending data to it (stdin/stdout etc) simplifying the UX (especially with feedback) on this would do a whole lot to make programming better. especially if we specialized a structure editor to understand not merely “here’s a semantically valid edit” but “here’s the environment before and after this change”
m
sounds a lot like unison, have you seen it?
k
Is there any reason why you said 'database' rather than just 'key-value store'? Environments don't need foreign keys or complex joins. They also tend to be pretty schema-less. Simple Lisp interpreters often implement environments directly out of a-lists, so a key-value store is an extremely natural analogy to this Lisper. Though perhaps a better analogy than either is a graph database, since lexical scopes can nest.
👍 1
g
The tricky part with any API is only requiring the necessary details. These implicit scopes benefit from not being seen at all times, though debuggers show it's valuable sometimes. In OOP, you bundle up all the information an object needs in properties or dependencies on the object. In FP, it's usually convenient once you get past 3 positional args to start bundling up related data in data structures, but functions can be only concerned with the parts they need and you write some intercessor layers to hide the rest.
Though I have found it valuable as a thought experiment for myself to make an analogy of well-factored FP code to DB normalizing/indexing. I learned the FP stuff first.
i
Yes it is! We need an environment with a better UX. Radically simple way is to just edit trees and connect them in graphs that cross multiple domains - a thing you defined in your db model is connected to the altered representation of the same thing in the UI model the user sees through a chain of transforming actions. Just defining your architecture and structures as trees with code generation can get you to great lengths. This is what my project is about, trying to create the simplest and best development tool, a simple list editor 😄
❤️ 1
v
@Ian Rumac Hey! I have similar thoughts. Is there any place where I can read more about your project?
g
reminds me of 'incremental', essentially a spreadsheet.
i
@Vladimir Gordeev not yet, I hope I can get an alpha up on gitlab in a month or so
g
@Mariano Guerra definitely! i love unison. they seem to be heavily focused on the idea of hash-addressing identifiers, which i think is critical for getting the backend to work—i’m thinking more about the UX of edits to the environment than how we track those edits with code
@Kartik Agaram that’s the half baked part haha. really just thinking database because (for me at least) it draws to mind visual/tabular interfaces in addition to pure text ones. i think you’re right about the graph database, or maybe more like datomic’s model where you have an entity attribute value transaction (and one other i can never remember) associated with each change (maybe plus some features like “changed by this function, reassigned by this function” for provenance
@Gary Trakhman i think the nesting stuff actually fits better with a db-style paradigm, or whatever it is that i’m describing—like with lenses, you might focus in on one data structure, or collapse it as you might in an outliner program
@Ian Rumac please also ping me when you put it on gitlab
k
In my experience the Datomic model is elegant but not to be gotten into lightly. The performance implications have to be explicitly designed for. It may be easier if the programming model is mostly immutable bindings.
g
@Gary Trakhman yep like incremental with an alternate ui—and maybe even slightly different semantics—if the system is observing every mapping by default, then you don’t need to “watch” values as much as let the system know this block of code depends on it, which could be detected by merely following the references (i think)
@Kartik Agaram yeah i’m being more metaphorical/evocative than advocating for a direct copy. but the idea would be to cover a similar resolution on data changes
g
put some VR on it 😄
would love to pull apart a computation graph with my bare hands
at least once
k
Bumping back to OP: Is this a change to the programming model, editor UI, editor implementation data model, or a compiler technique? 🤔
g
hahaha there’s a lot of that going on
@Kartik Agaram realized i was not clear: editor UI (but would have to be integrated with a programming model, maybe a la smalltalk)
k
Editing to 'programming model' inside my head. I tend to think of 'Editor UI' as "can be applied to any programming model". You want to display not just the change to an environment but also the current bindings at any point in the program, right?
g
like my example might be something like an up to date version of what self did:

https://youtu.be/5Jhi5yN9S1o

(even just the first minute or so where he creates a balance slot on an object and sets it to zero, and it becomes like a table object in the view). where like you have a straightforward gui/keyboard shortcut system for editing the environment instead of slots on an object
❤️ 2
d
This topic sounds similar to my goals for: https://github.com/d-cook/Interact
g
yes it’s EXTREMELY similar; i love the drawing metaphor
b
Perhaps https://scalameta.org/docs/semanticdb/specification.html is an angle on a similar idea? Its a "data model for semantic information such as symbols and types about programs."
d
I have many ideas/goals for [Interact](https://github.com/d-cook/Interact), but one is that instead of representing "source code" as static text, it is instead an entire interactive model/representation. And everything that makes that model work (how it is visualized and interacted with, how it builds, etc.) is built in to the self-same model. The relevance here is that everything about language (e.g. compilers, interpreters, syntax, semantics), data representation, visualization, editors/tools, ... everything about programs and programming, really ... It's really no different from (e.g. can be accomplished via) ordinary programming techniques using functions + data-structures + abstraction, etc. And since code can be represented as data-structures, the whole world of programming can fit into things like lists and key-value stores. You really don't need anything more than that for anything! As for the DB, you can code up whatever model works best instead, or you have something that translates your "ideal model" into code that creates and populates a DB and the code interface to it. (I can explain more if needed)
k
@Dan Cook can you talk about how your different projects are connected?
d
I'll start with "Objects": https://github.com/d-cook/Objects My first attempt/iteration at making a system entirely out of "objects" (e.g. JSON/LISP-like data-structures). Expressions are LISP-like (list of operation + arguments), but the interpreter/evaluator simply looks up ALL operations in a "context" object (which represents lexical scope and may reference a "parent" context -- think Scheme). Functions can be native, or consist of objects that contain the code to evaluate, the arguments it takes, and the context (scope) in which to do so (typically same as lexical scope of the function). The entire system (evaluator, a compiler, base operations like get, set, +, -, etc.) are all defined in a "root" context, which is not different from any other manipulatable entity. The evaluator was originally written natively (JavaScript for now), but then rewritten in "objects". The goal was to rewrite everything that way, and have code that regenerates the native code for the whole system which, when run, recreates everything as it was at the time. This mostly just means spitting out the static data-structure for the whole thing, inserting a native evaluator (by feeding it into the compiler) which is then invoked immediately on some code to bootstrap the UI. From that point, you can reshape the whole thing just by interacting with the running program -- and that will allow me to evolve it further without ever looking at the "source code" again! The only remaining "native" code (other than the base operations) was the logic in the compile function to convert code to JavaScript. But hey, I could (using the tool itself) make a compiler for something else (e.g. assembly, .NET, JVM) and supply the level base operations for it, and then boot up the entire system (in the same state) on something else! I took it too far with how minimal I wanted the native set to be (e.g. defining "lookup" in terms of objects, when that's a key function that the compiler injects everywhere), and forcing a CPS model under the hood sooner than I should have. It just became very difficult to manage, and stuff started to blow up when I got 90% of the way through rewriting it in itself ... I might go back some day and replay some of my development with different decisions. I'd been developing and interact with this entirely though the browser debugger, leaving the UI as an afterthought for after the base system was working properly. I knew I'd need some primitive graphical operations that could translate to any system -- just whatever is minimally sufficient to bootstrap a minimal foot-in-the-door UI, because then I could just use the UI to improve the UI code. ... But as stated, I never got to the UI. My "Interact" project starts from the UI first and will work backwards to eventually bootstrap itself into that UI. It's essentially becoming a do-over of "Objects", though I might come back to that model eventually and reconcile them. ... I'll describe "Interact" next
"Interact": https://github.com/d-cook/Interact (continuing from previous) ... So the minimal UI would just expose the raw structure of everything, and provide a way to execute / evaluate code. (That's not the end goal, but I'll get back to that later) Entities (strings, numbers, lists, functions, etc.) sit freely on a canvas, where they can be moved around, inspected, expanded/collapsed, etc. Dragging values onto a function causes the function to be evaluated with those "arguments", and the result appears as a new entity on the canvas. Since there is a function for every basic operation, this is sufficient for making every kind of manipulation possible (but not necessarily ideal - I'll get back to that!) (Certain basic things like getting or setting a value could be as simple as drag-and-drop; but I'll get a working tool first, and then use the tool itself to see what works best for what -- let's call that "phase 2") Either way (e.g. whether the UI "helps" or not), any sequence of operations that can be done -- whether through user interaction or by evaluating "code" -- directly corresponds to sequence of (nested or otherwise) function calls. Therefore, by recording and replaying actions taken, all code can be created and viewed as direct interactions, rather than as the textual or structural representation one normally thinks of as "code". *(From here I might use "code" and "actions" interchangeably) So essentially you can do a bunch of manipulations, and then say "See what I did? Call that a function, and THIS value here is the result". And then edit the *actions and/or the object that represents the new function to replace certain values with arguments, etc. Now whereas "normal" code references values by identifiers (e.g. looking up variables by name), the user is able to just "grab" values that are just sitting there. So under the hood, code is stored as a list of actions, which contain either simple values or the index of some other action (referring to the result of that action) ... In other words, it's a DAG. When viewing / editing a function, actual values are substituted from the arguments, and you are shown the "aftermath" of the actions (i.e. values on a canvas), as if you had performed all the actions yourself. Changing any actions or values causes the whole thing to recompute (think FRP) This means that any operation you do becomes part of the current "function" you are editing. I'll provide a way to "cut" dependencies though, so you can use the canvas as an interactive REPL, and then either throw out the results, or "forget" how it was computed and just use the generated value directly. I could also have a "cleanup" that will throw out all intermediate operations that did not contribute to the ultimate result of the function. All editing is done in a function, so even the "top level" (global scope) is a "function" with its own record of actions taken. This also means that all the work of "programming" you do is recorded as some step somewhere, giving you multiple layers of undo/redo. Anything you do can be abstracted into a function and edited for generic reuse. E.g. "Here's how I do x to the code" ... Anyway, I'm getting lost in low-level specifics. I'll tie this up in another reply
❤️ 2
(Continuing about "Interact" from before) Anyway, the point is not to have that specific tool per se, but to have something sufficient to edit code in a more freeform way, and to let whatever that looks like be very easy to evolve and change to whatever is more ideal for whatever context. The plan is to eventually bootstrap all the code for the tool into itself, so it can be changed and explored over time (per use, and to evolve this tool generally). The benefit is that whatever "nice way" is found to represent or edit something, can be immediately applied to itself to make it easier to make things easier, etc. One thing I'll provide to that end, is the idea of a custom "view". What I've described for viewing and editing functions is just one such "view", which is different than viewing a "function" as raw JSON. But perhaps something is better viewed as a table, or a graph, or some other interactive visualization. A custom view will contain whatever code is necessary to render something, and respond to interaction. This will probably "just be code", but (for example) there's no reason there couldn't be a custom view for specifying views using some custom DSL. And that brings me to another major goal of this tool: Things like DSLs, macros, syntax, even "programming language" (compiling, interpreting, translating), ... these can all be fluid things defined by ordinary code. The only difference is when and where it happens. But in a system like this, everything is the result of some "actions" taken to create it. So there's always some "code" that specifies how your code got created. And there's no reason to treat that code any differently! Expanding this further, with all the possible ways to view and edit "code" (or anything else), there's no reason to distinguish between "code" and a UI widget with its own custom behavior. No fundamental difference between a macro or a compiler, and anything else that let's you specify or view code/data one way, and then generate it another way.
... Sorry that took forever and is long-winded, but those are my main projects in ... something larger than a nutshell :P I also have a "Renderer" for rendering basic graphics in a declarative way (which I'm using for my other projects, but it's also generally useful). And a "Circuit" simulator I'm making PURELY just to demonstrate blurring the lines between code and interaction, etc., as I've described in my previous reply above
❤️ 2
m
I think it will clarify it more for me if you compare it with similar existing solutions, how does it compare to lisp (in a lisp machine)/smalltalk (pharo)/self
s
@Garth Goldwater re your original post - I've thought about this as well. I think of this pattern as the following metaphor: a person working through a math problem by writing it step by step on paper. The 'paper' is the external medium that's static and 'just data'. The human is the processor creating the new data from the old data at every step and storing it back on the paper. We recreate this pattern over and over in the computer as well, whether it's database/appserver or the local data structure and code inside a procedure.
☝️ 1