Regarding "the programming environment is primary, PL is secondary" (see tweet): do you consider yourself to be working on an "environment" rather than "just a language"? If not, have you considered working on an environment? What is preventing a more balanced approach (assuming this is indeed a good idea)? https://twitter.com/stevekrouse/status/1115569100279951360

I am not sure I understand the question. I believe the environment is essential to software engineering. I also believe the environment must be seen as a language, rather than an agglomeration of features as it is mostly treated today. For example, Emacs goes in this direction. Here is a reasoning: When a person is presented with two presentations of the same content and they would yield different results in terms of the effectiveness of understanding that content, it should follow that the form is essential. It also follows that understandability is not only a property of content alone. That is the case with data. And as software is data, it is also the case with software. Moldable development starts from this assumption, and we validated it over the past ten years quite thoroughly (much of the validation is seen under humane assessment, which is a subset of moldable development). So, I am saying that only focusing on language and code, we ignore an essential property and this will always provide a partial understanding of our domain. I maintain that the environment is an essential subject of study that we did not address systematically until now. Does this address the question?

I was struggling to word this question, but it seems that a lot of projects in this domain are indeed "only focusing on language and code" under the assumption that this is the most important place to focus or begin, in contrast to your twitter discussion with @stevekrouse. So, I wanted to informally survey the group to see to what extent this holds here. Perhaps the consensus is closer to what you say, that the distinction itself made in this way is perhaps not that accurate or useful ultimately. But I'm interested to know if there are particular reasons why prototypes in this domain tend to be language & code focused, rather than starting from this apparently broader perspective of an "environment".

One hypothesis is barrier to entry. Creating a textual language is relatively easy (compiler/interpreter is WAY simpler than GUI). A language has a significantly lower barrier to adoption than an environment. New environments are going to have complex interfaces with a lot of expectations (“what, no vim keybindings?“). In that view, it’s not necessarily an indication of priorities---people may believe environments are ultimately the most useful thing to work on, but they work on languages because that’s more accessible to the individual programmer here (perhaps this would be different if teams were building these things than individuals?).

This is an interesting point @Will, I agree. What are the consequences if this is at least partially the case? Is it actually a tractable tooling problem? For example does it mean we need something like "Racket for environment design"?

I'm strongly on the "environment before language" side. My project is all about making better tooling / environment for creating and manipulating software, specifically by making the tool modifiable from within itself, still you can mold the best tools and environment yourself. (And that may include molding the tools for molding). However, like Tudor says, I also see the separation between language and environment to be superficial. For example, the idea is to have whatever representation is best suited for what you are making. That might be a table, or it might be code. The rules for converting that representation to running code is moldable -- so if that representation is "code", then you are (to some extent) molding an interpreter or a compiler, and thus a language. So even the concept of "language" is part of the environment. But I suppose you could say the reverse: that a view with windows and buttons and diagrams is also the "language" that a tool is presenting (or allowing you to present) something in.

Projects like Hazel, Subtext, Pharo, spreadsheets and notebooks redefine the environment to different degrees. This also makes them stick out as islands in the everyday environment of Unix - cant use your favorite text editor or source control. It is far simpler to just fit into the de facto environment because the scope of what you need to design is well defined and often smaller and you get to reuse all the 'wonderful tooling'. You are also subject to the same hard ceilings. One way I think of this is 'systems have affordances too'. How you extend or improve a system has a lot to do with how you're guided by concepts provided by the system. The notion of a 'text file based programming language' is one affordance provided by Unix. Others are files, processes, shells, shell commands and so on.

Also most of the people here have solo projects, so it's a huge challenge to build an entire environment. Starting with the language might be a great first step (MVP) and continue building more tools in following versions.

But you don't have to start with the language. Fortunately for us all, there's lots of options here. Most of them lead to failure, because it is a hard thing we are attempting. But it's good to have lots of choices. There's hope of finding some way out in the humongous state space. "Environment vs language" is an unnecessarily blunt way to think about this state space. You give up a lot of room for maneuver if you try to build the language completely before thinking about the environment, or vice versa. There's always an environment even if it's implicit. If you think of yourself as just creating a language the environment is some lowest denominator like Unix. But according to the recent poll, most people here are creating something totally novel, with no connection to Unix or other extant platforms. Focusing on the language is a particularly fraught activity for us. It's going to close us to many opportunities. So I find it more useful to fold 'language' back into the environment, and just think of the environment as a whole. How can I gradually accumulate a new environment (over many years) in a way that gets me a steady stream of feedback along the way? This framing avoids the hazard of over-emphasizing languages in our current culture. It also requires letting go of preconceptions of 'environment' like that it has to be graphical or involve IDEs. (This mindset is also why I pushed back against discussing languages in isolation here a few days ago. It's not just some administrative discussion to be relegated to #CEXED56UR. We're all building a whole new environment from scratch, and it's an immense challenge. There's no way to approach that without also thinking about the way we frame it in our heads, and constantly try to correct biases. Building something new and large from scratch necessarily involves debate of _philosophy_[1].) [1] Henry Ford's autobiography is startlingly philosophical: http://www.gutenberg.org/ebooks/7213

Since people program only in entire systems, one must always consider language and environment together. The graphical language people have sometimes no visible textual language, The Luna people are trying to make it bidirectional between graphical and textual, and then there are many projects which are primarily textual in nature, but have some wizards/helpers to make some part of the process less tedious and more direct. But the underlying language, whether it be textual or graphical internally, is the key thing; after all the environment is there to make it easier to make a finished product, and the final stored artifact is the goal, and the environment is typically not present in the final product. Thus the environment is providing scaffolding; temporary structure that helps in the development and testing. If you are trying to get products into users' hands as quickly as possible, the minimum you can build is the underlying artifact, and the tooling around it can wait until later. Many languages don't need more than a text editor for tooling to build something. Tools like Glamorous Toolkit are highly optimized for Pharo, and i think that to a great extent the tooling is often very specific to a language. For years people used Eclipse, now VSCode and Intellij Idea seem to be the top 2 environments, and they are fairly language neutral. The more radical the language improvement, the more radical the changes to the tooling will have to be, and i expect that many of the languages being built by the members of this forum will have great difficulty fitting into the conventional forms that VSCode and Idea utilize, which are basically refinements of Eclipse.

Software grows in an environment as a plant grows in an environment. You no more have to create the environment to contribute to programming than you have to ignite the sun to grow tomatoes. I think Steve is right that the environment is the thing, though, and the programming language is just one small part of the equation.

Some of us feel strongly that the environment should be part of the finished product, and have this at the core of some of our projects. But we certainly recognize the difference between real world practically with software systems as they currently exists, vs experimenting with redefining the very concept of software & programming & "computer". (My way of dealing with that is to make the "new world" thing, but then also use it to enhance the development/generation of "stripped down artifact" programs that the real world currently operates on)

Help! Beginner question: What’s a “language”? And what’s an “environment”? Is language just the syntax? Does it include semantics just for what’s part of the language? Or does it include libraries as well? Just a standard library? Or a set of default libraries that are shipped with an SDK? Or even all libraries, wherever they come from? What about a (language) runtime? And we haven’t even started talking about tooling yet, which I assume is clearly part of the environment…? Somewhere there I crossed from language into environment, and I’m not quite clear where I did. Maybe that’s why the popular opinion “You can’t do one without the other” seems fairly reasonable?

Great discussion, thanks everyone 🍿

Some choices in language design are more suitable for some features that the environment can provide. So really both things go together. Lamdu’s choice of a pure language (a language with a distinction between calculations and effectful processes) allows to re-evaluate your code as you edit it with no danger of executing unwanted effects (which you can execute with an explicit user choice). The trade-offs around some features like static typing can also change dramatically with the design of the editing environment (does it allow you to freely make type errors or are those contained in “Fragments”/“Non-empty-holes”?)

I enjoy this thread quite a bit! I do agree that an environment is not trivial and it can get expensive. However, I strongly believe that the form of the programming language can be the form in which we end up writing but does not have to be (and in fact, it should not be) the only form through which we read. Reading and writing are two distinct use cases that do overlap but are not the same. So, while having bidirectional manipulation between a graphical and the textual representation is interesting, in my view, it is secondary to the ability of creating contextual reading or writing views that are unidirectional. @Edward de Jong / Beads Project Glamorous Toolkit does appear to be created for Pharo, but even if it is not yet too visible, it is in fact a platform for IDEs. The way we approach the differences is that instead of imposing a unified presentation (such as a tree project structure), we offer multiple components out of which a specific experience can be put together. Some of the components will turn out to be more generic (such as an editor), but the specifics will matter more and we optimize for the specifics. I like @Stefan’s the

What is a language?

question a lot. Indeed, most efforts in this space look at language as the thing that has a syntax expressed as a BNF grammar and the associated semantics. However, when we look at programming as being made of multiple layers of abstractions, an API or a library becomes as much of a language (indeed, that is the basis for the idea of an internal DSL). So, an environment should cater for those as well, and when looked at the problem from this angle, it turns out that we have much more of those languages than the BNF kind. For example, a system built in Angular and one in React, even though they are both written in JavaScript, will have fundamentally different needs from the surrounding environment. This is an obvious case, but a system is made of dozens of such libraries and even more system specific libraries, APIs and frameworks. All these can be seen as languages. Those kinds of languages are not easily representable in a uniform way. So, if it is to be effective, an environment should not be architected around a unified view idea. @shalabh raises an interesting point regarding the isolation that an environment creates. However, there is no intrinsic need for the environment to be the same. Much like how a projectional editor is just a layer on top of the runtime language, an environment as a whole is a similar layer, too. And, nothing stops us from utilizing another layer to look at the same content. So, you can well use a plain text editor to edit a notebook, but it might not be the thing you’d like using for an extensive period of time. And this brings me to the wonderful link provided by @Dan Cook. I was not familiar with Houyhnhnms, but I do subscribe to that point of view. Indeed, any environment is the way in which we decompose our actions with which might end up affecting the software to express our thoughts. To that extent, we can look at an environment as a language made of visual and interaction operators that extend and co-exist with what we classically understand as programming language.

I've got a clear border in Onex between language and what you seem to be referring to by "environment" here. Language is the pure functional tree/graph/object reducing stuff, which is entirely domain-independent - there are some built-in hard-coded reductions for basic data types like numbers, strings, lists, etc as you'd expect. Environment is where domain types are introduced, which maps onto i/o. Here I have the more complex structure types like users, addresses, calendar events, documents, 3D objects, etc, etc.. So I guess I define the line by that: domain-independent = language; domain-specific (i/o) = environment.

Hot take: language is an environment

I forget where (maybe Magic Ink or Learnable Programming), but Bret Victor talks about programming language (or code) playing the same role as a user interface (and when compared to other interfaces, it's a poor one)

When you look at the various competing next gen language projects: Red, Luna, Dark, etc. you have a language and a compiler or interpreter, a runtime for the language that permits it to run on a given operating system, and then various other tools, like lnterface builders, source code beautifiers, source code obfuscators, package/dependency managers, linters, cross reference tools, performance analyzers, post mortem/runtime debuggers, component webstores, deployment aids of various kinds, to see how it looks on various different hardware devices, so many different parts to a modern toolchain, much more than of yesteryear. This is one reason why so little progress has been made; one person can make a compiler pretty easily in under 1000 hours, which roughly equates to six months of one's time. But to build a complete toolchain, across multiple platforms, with documentation, and examples is really quite a big task. I would argue that the supporting tools and reasonably complete libraries for any new language is in fact a massive undertaking. This is why i think it is dangerous for people to start in the tooling world, because you can get lost in that vortex. New languages like Chapel, Parasail, and Red started with working compilers and are adding in the tools and documentation gradually, which allows them to grow organically. It is true however, that if you have the tooling available at the beginning, then it can launch more like a rocket, as it is much easier to work with when the scaffolding is solid, versus doing it the hard way. A lot of really complex systems were built without any good tooling; nothing more than print() statements in the old languages to go on; tooling just lowers the bar so that a person without superhuman patience can accomplish something. I am sure there are some grumpy old mainframe programmers who think that only real men digest core dumps that use up half a box of lineprinter paper.

@Edward de Jong / Beads Project, Clearly, many recognize such a span as what's necessary for a kind of freedom / power that should (and could) be possible with computers & software (else it would not be a topic with what "tends to happen"). But also clearly (as you expressed and I agree), that's not achievable with traditional methods. That's why much of that consideration comes from those of us exploring a "new world", which challenges /doesn't take the traditional idea of those things (e.g. "compiler") -- or even the entire pipeline itself -- as a given. I think your statement is a helpful wake-up call to those trying to go floor to ceiling with traditional methods ... and perhaps also evidence that they're what's holding back something better :)

With all the talk of tooling, this came to mind:

“If you want to teach people a new way of thinking, don't bother trying to teach them. Instead, give them a tool, the use of which will lead to new ways of thinking.” ― Richard Buckminster Fuller

Tooling and 'way of thinking' are linked (what kind of objects does your tooling operate on and what are the operations? these form the way of thinking.). E.g "editors edit *text files*" and "build systems automate *program compilation*" - the last two phrases reveal ways of thinking. One corollary is that as long as we're compatible with some tooling, we're implicitly adopting those ways of thinking. So whether we (a) build little custom tooling and mostly reuse existing tooling; or (b) build a lot of custom tooling up front, stay compatible with some existing tooling; might reveal how much of a new way thinking we are adopting. Even (b) adheres to existing ideas to some degree. There might be another option (c) replace the existing tooling completely and become incompatible with it. Maybe this brings very different ways of thinking. Anyway, that's where my interest lies. @Dan Cook’s post about two approaches seems related: https://github.com/d-cook/SomethingNew/issues/24#issuecomment-394217045 Also I saw this twitter thread put it very nicely as tendencies: 'situated' or 'radical': https://twitter.com/flippacpub/status/1115287533477355525

In the famous "Mythical Man Month" book, which i hope everyone has read already, Prof. Brooks writes about how a complete system is 3x the cost of building the raw tool. I would say this is a good estimate; that the tooling for a new language and the documentation is 3x the original compiler/interpreter effort.

Sure, but what if the "original" is already orders of magnitude larger than necessary? What if your "language" (or its compiler/interpreter) fits on a note card? This is not possible if you are just doing your own version of the same old stuff. Alan Kay states that if you print all the source code for MS Windows into textbooks and stack them up, it will be as tall as the Empire State building. As a proof of concept, he developed a fully functional operating system in 20K lines of code. (I know of single UI widgets on webpages that occupy ~100K lines of code!) That was only possible because "fully functional" was redefined purely in terms of usability, whilst throwing out most preconceived assumptions of what must make-up up an OS.

Prof. Wirth of ETH built his own computer, the Lilith machine, that had its own operating system, and was very complete and usable. It was amazingly compact, all written in Oberon, which was a successor to Modula-2 (which was the 10 year successor to Pascal). Not enough people appreciated how clever that was to make a whole computer and OS together.

Thanks @Dan Cook., Just watched the video. I believe the speaker is Faré, author of ngnghm.github.io.