https://futureofcoding.org/ logo
Join Slack
Powered by
It seems like a lot of you guys have looked into s...
# thinking-together
f
It seems like a lot of you guys have looked into structured / ast-based editors previously or even tried to implement an editor. I'd be interested to hear your thoughts about why these kinds of editors haven't been widely adopted. What are the challenges (both technical and social), what are the open questions, ...?
y
The experience in text editing, which we call “WYTIWYS” for “What you type is what you see” is “natural” and does not require an additional learning curve. Many structured editors diverged from WYTIWIS and I believe this has hurt their experience. Some exceptions to this are Lamdu (which I’m working on) and MPS which do attempt to retain a WYTIWYS experience.
s
The classic answer is that they are not fluid enough. Computers are especially designed for text editing (keyboard and mouse) so it takes a lot of effort to compete. The most successful one seems to be embeddr. In their papers, they have good analysis of the challenges and some solutions
y
btw iiuc mbeddr is based on MPS
Another challenge is that none of the various popular UI frameworks are really well suitable for creating code editors or rich keyboard oriented interfaces. That’s why we’ve needed to create our own UI framework for Lamdu.
d
One issue is the goal of "always be valid" - I don't think it's even desirable and makes such tools very poor creative tools. When you need to refactor something from structure A to structure B it's almost essential to go via invalid transitional stages. Using any non-text editor I inevitably need a workflow of 
export to text -> mangle -> import text
.
👍 1
💡 3
f
Thanks for the responses! @stevekrouse I'll take a(nother) look at mbeddr. @yairchu Could you describe what popular UI frameworks were missing and what you did differently in your own one? @duncanawoods That's right, I've had these kinds of problems before too...
y
@Felix Kohlgrüber A few things a UI framework needs: • Stuff you’re used to having in text editors, such as moving the cursor from the end of a long line to a shorter line and then moving it back and expecting it to be at that spot (and not at the long line in the position of the end of the short line) • Responsive layout suitable for laying out code/ASTs without horizontal scroll • Animations so that the user can easily comprehend what happens in refactorings (such as “extract to let item”) and layout changes (if the layout is responsive and automatic)
👍 2
j
If text is really such a natural way of working why don't we use it more for non-programmer UIs? It seems to me we've all spent so long learning the skills associated w/ text editing it feels natural, and even if there was a "more natural" UI devised in which you could be as productive with half thee training the existing set of programmers would have to re-learn a great deal of their skills
n
I'd recommend everyone to check out hazel.org as an example of a structured editor (/research project) that aims to solve some historical problems. In particular, it formalizes the notion of incomplete programs (programs with holes) for structured editing so that not only can you manipulate arbitrary fragments of code (as with a text editor), but you can also run them and they'll compute the fraction of the result that there exists code for. The current demo of Hazel is a bit basic and isn't necessarily representative of the end-goal, but it's under heavy development. @Cyrus Omar
☝️ 4
d
I'm designing a structured editor for text-based coding, where tapping/clicking any text element brings up a context-sensitive menu for editing the text element and/or for spawning additional text elements. See https://www.slideshare.net/secret/4AG6fsd0t3tCNi if you're curious.
y
@Nick Smith have you checked out Lamdu? Basically your very description of Hazel applies to Lamdu as well, except its current state is slightly less basic because we’ve been working on it hard for quite some time
n
@yairchu I have, and it's pretty cool 🙂. I would still consider them distinct, and I'm sure they'll make different design decisions as they grow.
j
Perhaps not enough attention has been put into user testing and UX when creating structured editors?
y
@jaukia it takes a lot of work to make a structured editor with a good enough experience to “be worthy” of a good user/UX testing. Note that MPS/mbeddr have actually done several formal user studies - [example](https://www.reddit.com/r/nosyntax/comments/93317x/towards_userfriendly_projectional_editors/) For Lamdu we’ve made some informal user testing and some extensive personal dog-fooding (writing apps with the system), and we got invaluable feedback which made us improve Lamdu a lot by going for WYTIWYS and working on good responsive layout (at first big programs required horizontal scroll which we found is horrible)
👍 1
d
@Jake Brownson it seems to me that we do use plain text for the closest thing to programmer UIs: spreadsheets and instructions from humans to other humans (pictograms may be used for the latter, but only in certain cases, and I think the same applies to graphical programming - it's useful only sometimes.) And note that the non-text disk format of the spreadsheet thwarts diff tools.
d
Something I experience which is quite subtle... kind of related @yairchu and WYTIWYS: It's mistrust. It's very psychological and hangs like a cloud of unease over all high-level editors including e.g. wyswyg word processors. Higher level editors are buggy and have invisible state that comes back to bite you in unpredictable and frustrating ways. This applies to basic things e.g. I still can't rely on bullet point lists or indentation in Google Docs. In contrast, text feels like a raw material that can't lie to you. The ground under your feet is solid and there won't be any nasty surprises. Very few high level editors get to the point of feeling "safe".
👍 5
@Jake Brownson
If text is really such a natural way of working why don't we use it more for non-programmer UIs?
I've thought about this too. I think for a constrained task, a constrained high level UI is the best. A lot of end user UI is task specific so that is the better choice. For creative tasks however, the use case is more like putting a lump of clay on a potter's wheel and then massaging it into your end result with the ability to explore lots of different paths. If a high level editor adds constraints and abstractions that means you can't do this, it becomes a bad editor. I am quite interested in the philosophy of craftsmanship which has a lot to do with interaction between a craftsman and material. I think a structured editor can achieve the same certainty as text if it's fundamental unit of material has the right properties. I'm not sure what these properties are but it's something like: - predicatable rules of combination - always possible to add or remove anything - no hidden state - can act on groups as you can on individuals Spreadsheets work quite well with cells as materials. It's not perfect but you know where they are and how to combine them so it feels solid. Spreadsheet formulas however, eh, not so much!
👍 1
b
Hypothesis: 80% of the benefit of structured editing is not the structure per-say, but in the shift in mindset such that you should provide tooling right next to and spatially integrated with the display of code—refactoring tools, debugging info, performance info, live feedback, related code search, etc. There’s no reason you can’t provide the same advanced tooling for text buffers—only that text buffers lock you (the IDE designer) into thinking about text manipulations and thinking that you are locked into a rigid display of the program. The value of the structured editor is that, because you are rethinking the code display, you are freed up to offer rich program insights and manipulations close to and interspersed with the code. It’s those tools, not the structural tree manipulations, where the real value is.
👍 2
❤️ 2
d
An issue with “classical” structure editors is the cursor UI, where the thing that is selected at any time is a subexpression (i.e., some subtree within your AST), and any edit you make targets that selected subexpression. This causes awkwardness in situations where we’d rather have text-like entry that relies on operator precedence to construct the tree appropriately. Say we want to type 
2 * 3 + 4
. We get to 
2 * [3]
, where 
3
is selected because we just constructed it. Then we hit plus. A simple structure editor implementation would lead to 
2 * (3 + [_])
, which is not what we want. In this implementation, we’d have to first navigate up to select the multiplication expression (
[2 * 3]
) and then hit plus. This kind of mode switching is annoying. http://www.cs.ox.ac.uk/bernard.sufrin/edit.pdf gives a solution to this that leverages online precedence parsing on insertion commands. Hazel does a form of this, I believe Lamdu and mbeddr do as well. Independently of whether we’re switching modes, AST navigation via ‘parent node’ and ‘child node’ commands can also be at odds with an interface that presents things as a linear sequence of characters.
👍 5
I think another fundamental issue with structure editors today is the implicit assumption that constructing an expression node needs to happen “all at once”. Suppose we have an AST structure that represents a parenthesized expression as a parentheses node with a single subexpression child node, and say we want to place 
1 + 2 + 3
in parentheses. Because we’re creating a single node, it seems to make sense that this should be a single ‘parenthesize’ command. This requires the user first to select the entire expression 
1 + 2 + 3
, then do ‘parenthesize’. So this requires the expression selection cursor UI, which has issues as described above. One thing that’s nice about a text interface is you can break up expression node construction into multiple insertions. Given 
|1 + 2 + 3
, I can type 
(
, then move over to the end of the desired subexpression, then type 
)
. One approach I’m exploring and implementing in Hazel is having an expression construction mode, where you can similarly rely on multiple keystrokes to specify how to construct an expression node. Given 
|1 + 2 + 3
, I type 
(
. This gives me 
(1[)] + 2 + 3
in expression construction mode, where the right parentheses is selected for moving to the appropriate spot. I hit right, that gives me 
(1 + 2[)] + 3
. Hit right again, I get 
(1 + 2 + 3[)]
. I then hit enter or 
)
and that returns me to the regular edit mode. People like to hate on modes, but it’s not that different from selecting autocomplete suggestions, which people like. The important thing is for that mode to be made visually clear and to be automatically entered/exited at expected times. This is what distinguishes this expression construction / autocomplete modality from the classical structure editing modality described above.
👍 5
🎉 1
f
Thanks for all of your input, I really appreciate it 👍
n
@stevekrouse Can we migrate to Zulip now so that threads like these aren't crammed into a tiny sidebar? 😅
👍 2
s
I set up futureofcoding.zulipchat.com and you can login by resetting your password. Some of us tried it out (I don't know if you were one of us) but I didn't love it. The threads are nicer but the rest of the UI is a bit awkward. It's not clear to me either way
r
A quick addition regarding the supremacy of plain text: No one has mentioned Unix. The "Unix Programming Environment" has completely dominated the programming industry. Text files are a core tenet of Unix (e.g., https://en.wikipedia.org/wiki/Unix_philosophy). Unix is is just as completely unimportant in all other industries, so you don't have the supremacy of text files anywhere else.
w
@robenkleene If only Unix kept up with the Unix philosophy.
😆 1
👍 1
n
I agree strongly with @David Moon that fluent cursor is the pain point. I have done some structural domain specific XML editor and while not FOP thing strictly issue was there close too. When you are implementing your own solution you are usually not alone. System will bring it's own limited cursor and selection that you have to somehow tolerate. Just ideas for your example: don't use parenthesis directly but somehow add one middle step there. For example: you cannot create real parenthesis but only 'suggestions'. Like: 
( 2 * [{] )
-> 
( 2 * { 1 + 3 [}] )
-> 
( 2 * ( 1 + 3 )[] )
(suggestions will upgrade automatically as real parenthesis when match is found). Therefore editor is aware which parenthesis are connected and structure won't be completely broken in middle way. Or maybe selection itself can be considered by inputting that {} and adding node is separated concern.
s
All great points above. One thing I didn't see mentioned is integration with the entrenched tooling. This is as much a technical issue (git compatible) as a cultural one ("I wont give up my editor!").
d
@Niko Autio Ah interesting. So I’ve also been thinking about transient edit states, where the parentheses node isn’t constructed in expression construction mode until you move the delimiters and “commit” them. Your idea seems similar, but perhaps with the added benefit of not forcing you to think about where a delimiter (in your example, the right parentheses) should go until you’ve written the inner subexpression. Then again, a lot of people are used to bracket autocompletion, so I could see it being acceptable that you have a balanced pair of parentheses before constructing the inner subexpression. I also wonder what should happen in the case of Hazel, where every edit state has a live computed result, when you have a left parentheses but not the right one. For example, if I have 
2 * (3 + 4
, should I assume there is no parentheses and compute 
2 * 3 + 4
, or should I assume some default position for the closing parentheses and compute 
2 * (3 + 4)
? I wonder if there would always be such a reasonable default position.
@shalabh One way I could see structure editing start to get integrated with existing tooling is as an editor plugin. What you store in version control is plain text, but the plugin can parse the program and then provide a structure editing interface that is automatically generated from compatible language grammars. It would be sweet if you had a Hazel-like editor plugin that takes a compatible language, detects which parts of it are purely functional (as needed for Hazel’s current model of liveness), then overlays those chunks of code with a live computation cell, where you can see a live result getting updated inline as you edit that piece of code. Underneath, the plugin is just making appropriate text changes to the file and that’s what gets committed to VC.
🤔 1
y
@David Moon IIUC that’s kind of similar to Unison’s current approach
d
@David Moon I think you are right but it is also a potential disproof of the value of structure editing. This type of thing is quite easy to add to existing IDEs like Visual Studio but there aren't a lot of compelling use cases. What actually gets much easier with a non-textual ide? If I had such a use case, I would create myself a plugin! For example, the most common "extra UI" you can pop up these days is for refactoring. The simplest example might be Resharper bringing up a dialog box to rename all instances of an element. However, the standard VS method to do this inline as text is a bit slicker and lower mental overhead - it highlights the text you are renaming in a special way and then dynamically updates all the references as you type. So 🤷 - I have plugins for visualizing code but I'm a little mixed on having them inside the IDE as yet-another-window. There are times that spinning them out into their own app makes them easier to use and compartmentalizes activities a bit better.
n
@duncanawoods One potential target is for using multiple languages. Let's say you are building HTML with JS. How do you offer all the HTML candies for HTML and JS candies for JS when they are side by side without creating completely new monster like JSX. Such things is not maybe super common it's good to think how does it effect on development when mixing languages is easy? For example having general purpose macro language and commenting semantics would be nice. However I am mostly considering structural editing as base for more powerful end-user GUI:s at first. Developing GUI:s is huge time sink and for complex problems having text inputs, checkboxes and selections is simply way too limited.
s
Slapping on a structured editor on top the existing languages will only have limited benefit - this is because the current languages are designed from a completely different mindset. It would be much nicer to have a top-to-bottom redesign where the UI and tooling isn't designed after the syntax and semantics, but they are in fact all designed holistically. E.g. we can't slap on an 'excel view' onto Python. Any view cannot escape from the Python-ness of the concepts you're working with. I agree strongly with @Brian Hempel - it's not the structure but that the medium you use to interact is itself more powerful. Not sure this makes sense but I see the static-document nature of the programming medium as something that we need to look beyond.
d
My conception of this structure editor plugin goes beyond the editing mechanics. I guess I’m really imagining a full IDE-like GUI overlay with editor services acting on the syntax tree getting structure-edited. As @Brian Hempel was describing, this could be a richer medium in which the results of editor services interspersed with the code being written. Maybe this isn’t quite possible with today’s plugin architectures (could you do this in Glamorous Toolkit? haven’t looked into it much). But if you could (which sounds feasible both technically and culturally given editor trends over the past ten years), then you could have a rich medium of interaction but still rely on existing text-based tooling for persistence and version control. @shalabh I agree that you’re fundamentally limited by the underlying language and that there would be benefit in designing languages from the ground up with the UI in mind, but I don’t see why those richer interaction mediums have to be at odds with current VC tooling. Maybe I’m missing a point/example here.
s
@David Moon - you can decompose the idea of a program into 'language + UI views'. But this presumes you have an underlying language. Instead, a deeper redesign would be 'model + UI views'. Here text could be used as a serialization format for the model, but being a serialization format it may not be readable. Instead you'd always want to look at the model through the UI views. As an example consider if we had to fit spreadsheets into git (with fine grained versioning of cells). You'd want to preserve identity of elements across versions in any decent structural editor. So it's not at odds, and can be made to work but there's an impedance mismatch. You can still use git, by reducing to a versioned key value store. So 'git log' may be useful but 'git diff' or browsing the repo in github would be meaningless since you are looking at some serialization of a model where the optimum view isn't text lines.
2 Views
Open in Slack
PreviousNext
Powered by