In a visual programming environment, points (like,...
# devlog-together
i
In a visual programming environment, points (like, x/y[/z/...] positions) are as important as strings are in a textual environment. If a visual programming environment doesn't have fantastic affordances for working with points, I can't take it seriously.
k
What do you think of Ronin's
$xy
?
i
It's a "lisp on the left". I think it was the tool I had in mind when I coined that term. So I don't personally count it as visual programming. You aren't programming through visuals, you're programming in text while looking at and lightly manipulating visual output.
$xy is a good example. It's a string. You have to type
$xy
to be permitted to use the mouse. It's keyboard-first. The text is what decides which visual things are allowed to exist. I'm keen on the opposite.
a
do you count orca as a visual programming environment?
i
Yes — fantastic pull. I personally see Orca more truly "visual" than most things people call "visual programming" (Pd/Max/vvvv/etc), because in Orca you program by arranging things in space. 2d position matters. Sure, it's on a coarse grid, and it uses text characters as symbols, but I see that as mostly a minor aesthetic choice. Going deeper: my interest in points is about pushing toward some kind of bootstrapp-y / codegen-y / visual parser tool. I want a programming environment where I can work with things in space (a la orca), but also create new kinds of things-in-space, without ever having to work with text.
a
ok 🙂 so do you count orca as a text-based programming environment?
i
Super no! Less so than Max/Pd, for sure. For me, it really comes down to… how much does what you're doing resemble written language vs drawing, sculpting, crafting or some other activity that feels different from writing.
How about you? How do you like to carve up the design space?
a
I guess I'm in the Allan Pavio-style 'why not both' category. My feeling is once you're describing discrete computation in visual form then it's going to be a form of text. But that doesn't mean that it isn't also visual. They are distinct categories of representation, but support each other and are integrated in human perception.
So I agree that orca is more visual than max/pd, but I don't see how it isn't also very much text-based
I think the desire to try to escape text is an accident of computer science history.. We developed text from our evolved visual and navigational capacity, so text is more 'advanced', but then computer science took text as a starting point and had to develop visual representations on top of that. So we're left with the feeling that visual representation is more 'advanced'' and that we should leave text behind. Really though they are dual sensory channels that should support each other.
i
Yeah, I like the "why not both" framing. If I ever rail against anything, it's that I have to do graphics by writing shit like
dot({x: 3, y: 5}, {x: -1, y: 0})
. I'm starting from a place of "we're using text for things that already have a better representation in a different form — why can't we use that form?"
a
Equally I find it very weird when people say max/puredata is visual and not text-based, when text is everywhere
i
Yeah, Max visualizes the graph of relations, and changes the authoring affordances. So it's not the same as traditional text code, but it's not very different in the grand scheme of what's possible.
a
Yeah or maybe it's the max user that visualises the graph of relations by painstakingly laying things out in 2d as secondary notation
you could pile up all the text on one spot and it would work the same
i
Did you ever see / do you remember the

DeepUI concept video

?
The stuff about how people think / getting ideas out of your head is hogwash. But the fact that you're working with data in a truly nontextual form is what I'm interested in exploring.
This is what I was getting at with the original post. Working with strings in a DeepUI-like environment might be awful. (It might be great, who knows, nobody's done it yet.) But working with points in space would be so much better than
position = {x: 3.5, y: 20}
, at least for certain use cases.
And if you want to get to something that allows visual bootstrapping / "codegen", having good facilities for reading/writing objects in space feels like a superpower. Thus, point affordances are really interesting to me.
a
Just watched it.. Saw quite a bit of text, even in a game based on basic geometry that needs almost no layers of abstraction. A bit more complexity and the text would be harder to ignore, unless it gets entirely hidden by a tangle of wires
icons are text, logic gate symbols are text, check boxes are text etc
there's also a fair bit of text text in the interface
It seems like it would be wonderful if you could somehow described discrete computation just by drawing stuff, but that's because it is wonderful - that's what text is!
i
Not sure if we agree on the meaning of text, at least as it pertains to conventional code. I'm not saying we need to avoid letters and numbers. I'm saying we need to avoid lines of text that you read left-to-right (typically, sigh, i know).
a
oh yep lets put eurocentric views on text in the bin
i
If I want to represent where something is in 2d or 3d space, I want to do that in 2d or 3d space.
a
decolonise text 🙂
i
{x: no, y: colons!}
a
well I guess that kind of text wasn't invented in europe so I hastily take that back
anyway agreed I think we need to reimagine text
i
So very narrowly: when you're programming something that exists in space, I want the programming to happen in the same kind of space. That's what strings are, that's why they're good! But something like
{x,y}
isn't.
a
I stumbled on this today, sharing it mostly unread but the title seems relevant ! https://arxiv.org/pdf/2505.15327
ah no preview - Let’s Take Esoteric Programming Languages Seriously by Singer and Draper
i
this looks like my jam
o
But strings are almost never used as strings (in PL), technically only strings in double quotes are "true" strings, existing in their own space. Otherwise they are symbols with a meaning. that is why they are so powerful, they are abstractions by default.