Coding tools treat code as static structured text. UX design tools treat design as static vector-graphics with "noodles for swapping a 1000 static states" on top. Code is data that needs direct manipulation with instant feedback. That's what creative tools need to achieve.

Most code doesn't have a UI, how does a visual canvas apply in these cases?

Good question. I haven't considered whether a visual canvas would be useful outside of UI/UX work. The key principle is to visualize the output of the running program in a way that would allow you to directly manipulate it though a set of controls that are overlayed on top. Any changes the user makes using those controls then need to affect the underlying code, which in turn will refresh the visualization to provide feedback.

where do existing app builder / low-code tools fit into this puzzle today?

I assume that by app builder you mean no-code in general. I think the main problem with no-code is that it treats code as secondary. No-code tools invent their own bespoke data-formats/walled gardens in which the data you edit while you work is "their data" and not code that you own. You're editing abstractions they invented, e.g. here's a tree of component-ish things that our visual no-code editor understands. It's from those trees that actual production code is derived/generated. In other words, code isn't a first-class data structure or editing capability in a no-code tool, it's more of an export/build step. Low-code adds a traditional textual code editor to the mix, but that text editor still doesn't treat code and its dynamic output as data -- data that you can have an immediate connection to via direct manipulation. Hope that makes sense? It's like existing tooling categories side-step the actual dynamic materiality of code (that code itself is both data, and data that can manipulate/produce other data dynamically).

Sure, but how is what you are proposing different? It's editing a UI application in a UI, yea?

To clarify, the "direct manipulation of code" needs to be supported as a visual canvas that shows the running product you're working on. The design tool needs to work back from there to materialize any direct manipulations you've made as corresponding code changes in real time.

sounds like the same pitch low/no-code use

Low-code and no-code has vendor lock-in and you're editing inside a walled garden. Editing code visually means you own the code and can use your IDE side-by-side to the visual tool. See http://worrydream.com/LearnableProgramming/ on why UIs for code editing make sense. Also Historically people either had to edit text to code up a UI (which not a lot of people have the skills to do), or they had to buy into the vendor lock-in of no/low-code. What I'm proposing is a visual IDE that has the usability/accessibility benefits of no-code, but the openness and code ownership you have with a traditional IDE.

Working on a no-code editor right now that views code as data. One of the primary ideas that sets it apart from other no-code editors is being generic - very few domain specific nodes, more about making core programming concepts no-code. The eventual goal is to be an abstract representation of all the other node editors (and js libraries). Would love to build in a side-by-side editor showing generated code or code-like syntax. https://nodysseus.ulysses.codes

Interesting approach 🙂 Can you elaborate on what you mean by "views code as data" because you also mention "generated code" which I see as opposing the notion that code is the data being edited visually be the tool. It's the difference between "code is the data with the abstractions" vs. "no-code nodes are abstractions from which code is generated".

In nodysseus specifically, "code is the data being edited visually". The user edits a graph which is walked and executed by the runtime in realtime. At the moment, no text code is generated, but the visual language itself is close to code - i.e. more closely resembles the graphs in sicp than domain specific nodr editors. More generally, for text code files in most languages an AST can be generated. The AST is another way of representing the text. They are both forms of data that define a program. Having one gives you a way of creating the other. One of the things I'd like to play with for nodysseus is transforming the graph representation of a program into a text representation and offering some ways of manipulating that text. E.g. I might implement https://hydra.ojack.xyz/ <-> nodysseus so changing a nodysseus graph changes the text and vice-versa. At the moment there is a hydra binding (https://nodysseus.ulysses.codes/#hydra_example) but I do not do any back and forth.

That's awesome! Was starting to think that we were the only ones crazy enough to treat code as the source of truth for a visual coding tool 😄 We'll soon support the ability to switch between working with code-as-data (ASTs) on the canvas and code-as-text so you can pick the best approach depending on what you're trying to do. Not easy to create a seamless user experience for it, but we're making good progress.

Super interested in following your approach! What's the project name?

We're https://www.henosia.com/ I share various videos of the development at https://twitter.com/jimmeyer 🙂