What do people here think about t-expressions? https://srfi.schemers.org/srfi-110/srfi-110.html @Kartik Agaram I noticed your name in the acknowledgements, do you have any comments?

This looks like the best summary of my opinion: https://news.ycombinator.com/item?id=8503353#8507385 In searching for it I also ran into http://www.arclanguage.org/item?id=17084, which seems like a fun thread in general.

I think it is very difficult to create a new programming language syntax that, on one hand, is "homoiconic" and "general" (the goals for t-expressions), and that on the other hand, has the characteristics of a well designed notation, including suggestiveness, readability and usability.

My take on this is that if the goal of a syntax is malleability and simplicity, plain s-expressions are impossible to beat, an readability and usability can be ensured by tool support. All the proposed alternatives have failed so far because they didn’t start out with a clear alternative set of compromises. They want to better in some respect, but don’t say which price they are wiling to pay for that.

@Konrad Hinsen I'm curious to hear what you think of my proposal: http://akkartik.name/post/wart. I don't work on it anymore and the rest of the language has fatal issues, but I still think the whitespace and infix support was quite elegant.

One of my fundamental requirements for a programming language syntax is that values print as constructor expressions, which, when evaluated, reconstruct the original value. For example,

>>> 2 + 2
4
>>> "foo" ++ "bar"
"foobar"
>>> [1,2,3] ++ [4,5,6]
[1,2,3,4,5,6]

The above REPL session shows a language where numbers, strings and lists print as constructor expressions. I don't think this requirement is in conflict with simplicity and malleability. However, Lisp dialects (languages with S-expression syntax) do not meet this requirement for lists, although they meet it for numbers and strings. For example, quoting from the Wart paper,

1 + 1
⇒ 2

'(1 2 3)
⇒ (1 2 3)

Philip Wadler wrote about how this causes problems for teaching Lisp to undergraduates:

The difference between (1 2 3) and (quote (1 2 3)) is subtle, and it inevitably confuses students. In particular, it plays havoc with the substitution model of evaluation.

From "A critique of Abelson and Sussman -or- Why calculating is better than scheming" https://www.cs.kent.ac.uk/people/staff/dat/miranda/wadler87.pdf

@Doug Moen Yes! Quote is weird this way.

@Kartik Agaram Wart looks interesting in many respects, in particular the way it deals with name overloading and how it encourages forking. But the topic here is syntax, which I think is not explained in sufficient detail on your page. Apparently you are trading in simplicity for "normality" in the sense of allowing familiar infix expressions. The price to pay is a distinction between standard identifiers and infix operator identifiers. OK, but what are those rules? Is it by character class, as in Smalltalk? Some characters are only allowed in infix identifiers, and others only in standard identifiers? With mixing forbidden? Then... how do you handle the creation of synthetic identifiers in macros? Stuff like Lisp's gensym. Another detail that is unclear is the indentation analysis. What would be the parenthesized form of

def foo

foo bar

foo bar

(assuming this is legal syntax?) That's where edge cases tend to get messy in various indentation-based proposals I have seen.

@Konrad Hinsen Thanks for thinking about this! The rule for inserting parens is: Multi-word lines without leading parens are grouped with later indented lines. That's it. I work through some implications at https://github.com/akkartik/wart/blob/master/004optional_parens To tweak your above example a little to make it legal code:

def (foo)
    prn 1
  prn 2

Now

(foo)

prints:

1
2

---

The price to pay is a distinction between standard identifiers and infix operator identifiers. OK, but what are those rules? Is it by character class, as in Smalltalk? Some characters are only allowed in infix identifiers, and others only in standard identifiers? With mixing forbidden? Then... how do you handle the creation of synthetic identifiers in macros? Stuff like Lisp's gensym.

Yeah, disjoint character classes. The major restriction is that you can't use

-

in variables, which is a bitter pill to swallow. (Mu gave up Lisp to get back

-

🙂) I don't follow the gensym question. Wart expands

$x

inside macros to something like

x1234

. But it doesn't support infix operators. Is that what you're wondering? It never occurred to me to create new infix operators inside macros.

@Konrad Hinsen One of the nice properties of

[a,b,c]

syntax for constructing lists, is that you can replace any of the list elements with a subexpression that denotes the same value. This property goes by several names: compositional semantics, composable syntax, "the substitution model of evaluation". The corresponding syntax in Lisp is

(list a b c)

. So, print

'(a b c)

as

(list 'a 'b 'c)

. But that's very verbose compared to the usual alternative. In Lisp culture,

(quote a)

is so important that it is abbreviated as

'a

, but there is no abbreviation for

(list a b c)

. So let's make an abbreviation, and use

[a b c]

as an abbreviation for

(list a b c)

. So now we print

'(a b c)

as

['a 'b 'c]

.

@Doug Moen FYI Racket has the property you want:

> (list 1 2 3)
'(1 2 3)
> (list 1 2 (list 3 4))
'(1 2 (3 4))

I have ambivalent feelings about this 'feature', though.

@Kartik Agaram Yes, gensym for infix operators was what I was thinking of. But there is more:

(let ((fn (make-symbol "foo-bar")))

`(eval

(progn (defun ,fn () nil)

(,fn)))

This must raise an error at some point - but where? Do you disallow making mixed symbols? Using them as function names? Calling the misnamed function? My point is that in a language with Lisp-style meta-programming, any constraint on syntax percolates down to lower layers, with hard to foresee consequences. And that's why s-expressions are so hard to replace, in my opinion.

There's no error there. It's just more inconvenient to call. If you create the symbol using

make-symbol

(or

sym

in Wart), you have to also refer to it the same way.

(eval `(def (foo a b) (let ,(sym "a-b") 34 ,(sym "a-b"))))
(foo 8 9)
=> 34  # args are ignored

Does this seem like a big problem? I've learned to expect that metaprogramming is a sharp tool that can get one in trouble.

The rationale for Sweet-Expressions begins with "Many software developers find Lisp s-expression notation inconvenient and unpleasant to read." And continues with "Mainstream adoption will, however, always be curtailed by the syntax.". I think that Sweet-Expressions fails at the goal of finding a syntax that makes Lisp acceptable to the mainstream, and I think this failure is due in part to not having a full understanding of what the problems are with Lisp syntax (from a mainstream perspective). The proposal's author boils down the issues to (1) a lack of infix notation, and (2) too many parentheses, which can be fixed by indentation as syntax. This is not the complete set: the Philip Wadler paper I cited earlier discusses additional issues.

@Kartik Agaram So how do you print those symbols then? As (sym "a-b") as well?

No, they print as

a-b

. Construction happens after the read phase. Infix expansion happens during the read phase.

@Doug Moen That sounds like a good summary of the trade-off. I doubt it was a conscious decision for Lisp, more like a tacit priority.

I see. Cross-talk between Doug's goals and mine. Thank you!

On the topic of arithmetic and infix operators: I spent a lot of time researching this for my Leibniz project, which is all about mathematics, with lots of operators. Any identifier can be used as an infix operators. Initially, I followed APL and Smalltalk in adopting a no-precedence left-to-right evaluation (APL is right-to-left). But then Shriram Krishnamurthi pointed me to the approach he had taken in his Pycket language (designed for teaching), which I adopted immediately and so far without regrets. The rule is simple: an expression with more than one infix operator requires parentheses for disambiguation. So, no precedence, nor left-to-right. The one exception is a chain of identical infix operators. So you can write

2 + 3 + 4 + 4

, which is parsed left-to-right. The net outcome is that you can read expressions as if they used traditional maths rules, but you don't need to remember precedence rules. And due to the same-operator exception, the number of parentheses remains very reasonable.

That is indeed clean and easy to explain.

Thanks Konrad. I'm looking at Leibnitz right now (http://khinsen.net/leibniz/). A small correction: Leibnitz borrows its infix precedence rules from Pyret (pyret.org), not Pycket.

Indeed!

thanks for all the pointers after reading up some more, and trying out some of the proposals, I'm back to just regular S-expressions, though I must admit the infix rule @Konrad Hinsen mentioned is very elegant and is something I will remember. I suppose there's still always room for macros that change the syntax (I like LOOP, please no flames), and going even further, stuff like racket's #lang. My main issue with syntactic extensions and changes, is that they usually break tooling support—usually macros don't provide too useful error messages, and static analysis tools don't work on the files anymore. Even basic stuff like syntax highlighting. Does anyone know of work in this area? I know of racket's syntax objects and still need to dive deeper into how they're used, but I'm not aware of much else.

@S.M Mukarram Nainar Racket is where all the action happens right now on this topic. The PLT group is seriously thinking about a non-s-expression-based language for their ecosystem (codename "Rhombus": https://github.com/racket/rhombus-brainstorming). Racket's syntax objects are extensions of Scheme syntax objects, which are lists plus metadata for tracing the provenance of transformed code back to the source code files. This metadata is required for implementing correct lexical scoping for identifiers, something that Common Lisp (and others) don't care about. It's a double-edged word: correct scoping is certainly necessary to scale up macro processing to the point of supporting complex language implenentations, but it also increases the learning curve significantly. Common Lisp macros are much simpler, and much easier to understand, but not at all easy to use for complex transformations.

@Konrad Hinsen interesting. I will have to spend some time learning about them. Is Beautiful Racket still the recommended resource? On another note, github.com/mflatt/rhombus-brainstorming/blob/shrubbery/shrubbery/0000-shrubbery.md is quite relevant to the earlier discussion.

I just delete the attachment in these situations. There should be a 'x' for you up top on the web interface.

@S.M Mukarram Nainar Beautiful Racket is a very good entry point. There are also many little languages by now in the Racket ecosystem that are good examples to study.

I like sweet expressions - they are similar in spirit to my own Loyc Expression Syntax (loyc.net/les), and I see they've been enhanced since I last saw them (it was a very simple proposal originally). Are any major tutorials teaching a Lisp variant based on them?

@David Piepgrass Nothing I know of, though a tutorial for Lisp newbies would probably be the best way to gain an audience. Selling any alternative to s-expressions to people who are perfectly happy with s-expressions is a pretty dumb marketing strategy.