Title
#thinking-together
Jim Meyer

Jim Meyer

10/16/2022, 11:39 AM
Coding tools treat code as static structured text. UX design tools treat design as static vector-graphics with "noodles for swapping a 1000 static states" on top. Code is data that needs direct manipulation with instant feedback. That's what creative tools need to achieve.
11:51 AM
To clarify, the "direct manipulation of code" needs to be supported as a visual canvas that shows the running product you're working on. The design tool needs to work back from there to materialize any direct manipulations you've made as corresponding code changes in real time.
Tony Worm

Tony Worm

10/18/2022, 6:34 PM
Most code doesn't have a UI, how does a visual canvas apply in these cases?
Jim Meyer

Jim Meyer

10/18/2022, 6:46 PM
Good question. I haven't considered whether a visual canvas would be useful outside of UI/UX work. The key principle is to visualize the output of the running program in a way that would allow you to directly manipulate it though a set of controls that are overlayed on top. Any changes the user makes using those controls then need to affect the underlying code, which in turn will refresh the visualization to provide feedback.
Tony Worm

Tony Worm

10/19/2022, 8:04 PM
where do existing app builder / low-code tools fit into this puzzle today?
Jim Meyer

Jim Meyer

10/20/2022, 5:23 AM
I assume that by app builder you mean no-code in general. I think the main problem with no-code is that it treats code as secondary. No-code tools invent their own bespoke data-formats/walled gardens in which the data you edit while you work is "their data" and not code that you own. You're editing abstractions they invented, e.g. here's a tree of component-ish things that our visual no-code editor understands. It's from those trees that actual production code is derived/generated. In other words, code isn't a first-class data structure or editing capability in a no-code tool, it's more of an export/build step. Low-code adds a traditional textual code editor to the mix, but that text editor still doesn't treat code and its dynamic output as data -- data that you can have an immediate connection to via direct manipulation. Hope that makes sense? It's like existing tooling categories side-step the actual dynamic materiality of code (that code itself is both data, and data that can manipulate/produce other data dynamically).
Tony Worm

Tony Worm

10/20/2022, 5:25 PM
Sure, but how is what you are proposing different? It's editing a UI application in a UI, yea?
To clarify, the "direct manipulation of code" needs to be supported as a visual canvas that shows the running product you're working on. The design tool needs to work back from there to materialize any direct manipulations you've made as corresponding code changes in real time.
sounds like the same pitch low/no-code use
5:26 PM
why is editing from a UI more efficient than directly editing the text?
Jim Meyer

Jim Meyer

10/20/2022, 5:36 PM
Low-code and no-code has vendor lock-in and you're editing inside a walled garden. Editing code visually means you own the code and can use your IDE side-by-side to the visual tool. See http://worrydream.com/LearnableProgramming/ on why UIs for code editing make sense. Also

https://www.youtube.com/watch?v=PUv66718DII

Historically people either had to edit text to code up a UI (which not a lot of people have the skills to do), or they had to buy into the vendor lock-in of no/low-code. What I'm proposing is a visual IDE that has the usability/accessibility benefits of no-code, but the openness and code ownership you have with a traditional IDE.
u

Ulysses Popple

10/28/2022, 6:51 AM
Working on a no-code editor right now that views code as data. One of the primary ideas that sets it apart from other no-code editors is being generic - very few domain specific nodes, more about making core programming concepts no-code. The eventual goal is to be an abstract representation of all the other node editors (and js libraries). Would love to build in a side-by-side editor showing generated code or code-like syntax. https://nodysseus.ulysses.codes
Jim Meyer

Jim Meyer

10/28/2022, 7:01 AM
Interesting approach 🙂 Can you elaborate on what you mean by "views code as data" because you also mention "generated code" which I see as opposing the notion that code is the data being edited visually be the tool. It's the difference between "code is the data with the abstractions" vs. "no-code nodes are abstractions from which code is generated".
u

Ulysses Popple

10/28/2022, 7:22 AM
In nodysseus specifically, "code is the data being edited visually". The user edits a graph which is walked and executed by the runtime in realtime. At the moment, no text code is generated, but the visual language itself is close to code - i.e. more closely resembles the graphs in sicp than domain specific nodr editors. More generally, for text code files in most languages an AST can be generated. The AST is another way of representing the text. They are both forms of data that define a program. Having one gives you a way of creating the other. One of the things I'd like to play with for nodysseus is transforming the graph representation of a program into a text representation and offering some ways of manipulating that text. E.g. I might implement https://hydra.ojack.xyz/ <-> nodysseus so changing a nodysseus graph changes the text and vice-versa. At the moment there is a hydra binding (https://nodysseus.ulysses.codes/#hydra_example) but I do not do any back and forth.
7:30 AM
To address some other points: yes no-code vendor lockin is very real and it's super difficult to navigate. There are a lot of people who "don't code" but make very beautiful things in Houdini/Nuke/TouchDesigner/etc. They are afraid of moving to a new system because they'd have to relearn their ways of working from scratch. It's a shame because the general syntax of these from a code perspective is very similar - some inputs and parameters to a node that does work and outputs something. There is an abstraction that could be used to interface with all of them.
Jim Meyer

Jim Meyer

10/28/2022, 7:43 AM
That's awesome! Was starting to think that we were the only ones crazy enough to treat code as the source of truth for a visual coding tool 😄 We'll soon support the ability to switch between working with code-as-data (ASTs) on the canvas and code-as-text so you can pick the best approach depending on what you're trying to do. Not easy to create a seamless user experience for it, but we're making good progress.
u

Ulysses Popple

10/28/2022, 8:24 AM
Super interested in following your approach! What's the project name?
Jim Meyer

Jim Meyer

10/28/2022, 10:22 AM
We're https://www.henosia.com/ I share various videos of the development at https://twitter.com/jimmeyer 🙂