When designing a programming language, what are good resources for designing the error / exception system?

Perhaps https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node312.html

Algebraic Effects obviously! 😄

The process outlined in @abeyer's link — "capture an error, inspect the stack, edit and re-evaluate code, then attempt to continue from the same point" — is an amazingly powerful tool for the working programmer.

And I find kinda amusing that that's basically how Algebraic Effects work, but with static type system on top of that. 😉

@Alex Chichigin Algebraic Effects are completely orthogonal to what I've described here, which is a feature of the development environment rather than a property of the language itself.

@Jack Rusher funny enough "evaluate handler in the context of an error location and continue execution from that point" was first and foremost implemented in Smalltalk and Common List which support it on the language level. Other IDEs still struggle to implement this functionality in its full. 🙂

Use monads instead of errors

@Alex Chichigin Common Lisp and Smalltalk make no distinction between "language" and "development environment", being live systems. Put differently, they are more integrated than anything called IDE today.

Came here to echo @Alex Chichigin about Common Lisp conditions. Practical Common Lisp has a good introduction https://gigamonkeys.com/book/beyond-exception-handling-conditions-and-restarts.html. @Jack Rusher had a good tl;dr. More generally, I feel we underutilize the stack as a way to abstract. Instead of explicit passing, build context through selective use of dynamically scoped variables. Alternatively, I guess there's the aspect oriented notion of CFlow https://schuchert.github.io/wikispaces/pages/aop/AspectJ_CFlowExplained. And I guess better functional programmers than me could say something about comonads, but in as much as option, collection, either, and exception monads are all about returning things, there should be a sort of opposite construction.

Give an example of an error/exception that is meaningful in your language!

My washing machine has exception handling. If I overcharge it, it beeps and shows an error code on the display. Legal documents (including law itself) also have exception handling. There's often a description of the "normal" case, followed by special treatments for exceptional cases. I find the distinction between "normal" and "exceptional" useful in many circumstances. Some programmers overuse or even abuse exception handling, but overall it looks like a good way of structuring code for humans.

Hmmmm.. these are both examples of domain level exceptions, also known as "just normal programming"!

Indeed. But without an exception system, normal programming means having the exceptions appear all over the code, either as explicit tests at all levels of abstractions (as in good old C), or as messy types (e.g. monads in Haskell).

@Konrad Hinsen curious why you think monads are messy

@Tony O'Dell It's not monads as such that are messy, but their use as a substitute for exception mechanisms. In each function at each level of your software stack, you have to consider and possibly handle exceptional situations, at the very least by wrapping your types in

Maybe

or some more specific monad. That's a lot of code clutter.

what's your take on golang's mess vs the monad pattern matching? the former gives you a weird mess of if/thens, the latter provides a little clearer picture (i'm not advocating haskell).

data, err := web.Get("<http://someservice.com/health|someservice.com/health>")
if err != nil {
  if <http://errors.Is|errors.Is>(err, ErrNotFound) {
    ...
  } else if <http://errors.Is|errors.Is>(err, ErrTimeOut) {
    ...
  } else {
    ...
  }
}

vs

case WebGet("<http://someservice.com/health|someservice.com/health>") of
  NotFound => ...
  TimeOut  => ...
  _ => ...

Pattern matching is certainly a bit nicer than conditionals (though it's not clear to me if the increased compiler complexity is a worthwhile trade-off) But I imagine both seem messy if you're anchored on data transfers that skip function calls on the stack.

seems like complexity might be conflated with unfamiliarity. i think there's a worse tradeoff made with the if/else handling in that the compiler can't enumerate the types of errors or ask the user to consider them

@Tony O'Dell Your Go example looks exactly like error handling in C. Which is messy as well of course. And, as you say, worse in terms of what the compiler can do usefully. But my argument is based purely on the human aspect of code, and the human way of describing exceptional behaviour is via a separate description of exceptional cases, separate from the normal path. Example: you describe to a friend how to drive to your house. Turn right there, then left there, etc. Do you add, at every step of your description, "if you have an accident, call the police?" I bet you don't.

This is an excellent point, but of course if you looked at a map and it only had the road drawn on it that you needed and nothing else it'd be a rather difficult map to read

I don't follow what your analogy is trying to say, @Tony O'Dell. I'm still thinking about @Duncan Cragg and @Konrad Hinsen's exchange about domain-level vs system exceptions. It does seem a useful reasonable goal that lower levels of the stack not throw exceptions (same as "require applications to specify error handling at all times"?) It's also useful to provide a back-channel for data flow that better fits the way a single app's developer is likely to think. The trouble often is that, having provided an exception mechanism in system software, there's a temptation to use it for system software. In the spirit of self-hosting, dogfooding and so on. Perhaps that's the problem? It's good to give people the ability to demote some code paths, that they can use to taste. But it's also good not to require the use of those code paths in libraries. I wonder if this is the nub of the disagreement every time the subject of exceptions comes up (ever since https://www.joelonsoftware.com/2003/10/13/13) On the subject of Common Lisp's interactive condition system, do people have any links to resources for learning more? Particularly tutorials with working code. A long-standing frustration of mine with Common Lisp: extremely low hit rate when I try to type in code fragments from blog posts. More than other languages, Common Lisp blog posts tend to assume a baseline level of knowledge -- about asdf, editor setup, what to load from a file, what to type in at the REPL -- that I've never been able to attain. (Most recent example of this: https://pvk.ca/Blog/2014/03/15/sbcl-the-ultimate-assembly-code-breadboard. Though now I notice it's from 2014, so never mind.)

@Kartik Agaram The most detailed description of the Common Lisp condition system is probably Michał Herda's recent book: https://www.apress.com/gp/book/9781484261330

Whereas reality has a lot of detail[1]; and Whereas anything that can go wrong will; and Whereas exception handling policies streamline program business logic; and Whereas code written by others becomes more difficult to debug in the presence of exceptions; therefore be it Resolved that the flow of exceptions will never span ownership boundaries; and further Resolved that libraries will never throw exceptions visible to their callers. So decided by the society of mind of Kartik Agaram on this ninth day of September in the year two thousand and twenty one. [1] http://johnsalvatier.org/blog/2017/reality-has-a-surprising-amount-of-detail

A bit of a personal opinion on the infamous Monads (as that translates to other effect systems too). Yeah monads just for error handling is an overkill (in the terms of code boilerplate though do-notation does clean syntax quite a bit compared to Go-style manual error code checks). But. When you add other effects to the mix -- async, IO, transactions, business-rules DSLs and what have you -- monads become rather economic and handy unifying framework. Obviously through _compositionality and abstraction_: you can make any particular piece of code only dependent on effects it needs and completely oblivious to any other effects including ones that get added later in development. And I'd like to point out that many of the mentioned effects in Haskell are implemented in libraries and not built into the language. 🙂

@Kartik Agaram May I ask for an exception for the specific case of interactive coding tools (IDE or otherwise)? They should be allowed to access exceptions in the code they manage at any level down the stack, for testing and debugging purposes.

@Konrad Hinsen No objection! Though it seems difficult to reconcile. Exceptions won't be caught by the environment if they're caught by an intervening stack frame, right? It seems a difficult decision whether any given interface should return a value or throw an exception, and IDE features complicate this picture.

@Kartik Agaram With given languages and environments, yes, it's hard to implement. It should be doable with the Common Lisp condition system, if all code respects some conventions. And future languages could implement exception handling per component, rather than globally, and add a "supervisor" mode.

@Konrad Hinsen yeah, monads are not composable in the sense they don't commute -- that's just their mathematical property and arguably the nature of effects we model (there are well-known exceptions and the laws governing those exceptions). So when I say "monads" I mean "monad transformers" too because to me they become inseparable (and quite natural) part of the picture. I even put the free(r) monadic DSLs into the conceptual mix. So I guess for you the "Haskell effects story" became only more complicated. As long as this stuff is NOT built into the language but provided in libraries, there quite a number of such libraries implementing different approaches to effects composition including Monad Transformers-based (good 'ol MTL but also Capabilities and some others), Algebraic Effects-based, Free(r) Monads-based and maybe some other too.

Dahl has a good talk about how monads in haskell could be a lot better. (again, I'm not a haskell programmer) @Kartik Agaram my point with the monads/haskell (again, i'm not a haskell programmer) and the map analogy is that knowing only what the program does and not the options for doing that or the intent make debugging other people's code difficult. if i only know what it is doing (arriving at a destination) and not the options for getting there (eg if i only have a line on a piece of paper with the turns and destination but no landmarks, roads, etc) then suddenly i need to do a lot of reading/research instead of just taking another road when the freeway i need has construction.

@Tony O'Dell do you have a link to that Dahl talk about monads at hand? Sounds pretty interesting. 🙂

@Tony O'Dell I'm again left with the sense this analogy would make a fantastic top-level thread ("How can we illuminate a map of the neighborhood of a program in the space of possible programs?") but struggling to see how it connects up with exceptions. A map with a list of turns and destination but no landmarks, just alternative routes to the police station from each turn -- this wouldn't be a map either.

Thanks @Alex Chichigin for the update. I do understand the need for monad transformers (as opposed to plain monads) from a mathematical point of view, but I was hoping that in the meantime someone had come up with a nicer way of packaging all that - or a different mechanism altogether.

@Konrad Hinsen I don't think there's any other approach to composing monads specifically apart from monad transformers due to mathematical properties of the construction. OTOH there are other approaches to formalising effects (at least a big chunk of them), Algebraic Effects (and Handlers) being one and Free Monadic DSLs being another.

@Kartik Agaram but if we have no exceptions at all, what to do when dividing by zero, or running out of resources?

Abort? Though I'm curious what @Duncan Cragg had in mind.

But what about if it is a recoverable issue?

Isn't that a property of the runtime rather than the issue? A sufficiently advanced runtime can probably recover from any error. The question is whether that's a good use of resources and complexity budget.

Sometimes it is easy to check in advance whether I will likely run out of resources when trying to get an exact result, and then I can switch to the heuristic which provides me a decent estimate. But sometimes doing that check in advance can be arbitrary expensive, maybe even more expensive than just running the computation. In that case it is nice to just run the computation, and in case I run out of resources, recover and run the heuristic instead.

For sure.

The essay on "reality has a lot of details" was a fun read. I used to say "knowledge is fractal" when we were working on the data model for Wikidata

One way to implement GC is to trigger it when heap is exhausted. A copying collector runs when half the heap is exhausted, but mark and sweep can in principle run to 100%. Then abort if nothing could be reclaimed.

Currently if you've run out of memory or disk, applications usually behave very badly, failing silently or crashing, maybe occasionally expiring with a meaningful warning. It's usually the operating system's problem at this stage. So I don't think that's the best example of an exception!

As far as I understand current "industrial consensus" on exceptions/errors (disclaimer: I'm biased and not "deeply immersed" into industry) it goes like this. There are conceptually two kinds of errors: sorta expected and recoverable ones (file not found and alike) and unexpected and (practically) unrecoverable (OOM and alike). For the first it's advised to use just regular return values, preferably wrapped into an ADT (aka enum) like `Option`/`Maybe`, `Result`/`Either` occasionally custom one. For the second kind it's advised to just "panic" (with optional mechanism to still handle the panic if you absolutely required to do so). Additionally I'm personally a fan of type systems that make effects explicit, in particular which code can raise errors/exceptions and which is pure (thus only prone to panics due to unrecoverable hardware/OS constraints).

Ouch. As far as I can see, that's never correct and often an unrecoverable mistake. Ideally, I'd want to special return values for integer division, one for 0/0 and one for n/0 with n != 0. If that's too much, make 0/0 equal to one and have only one special return value.

go's idiomatic tuple use handle the panic vs error thing fine, monads from haskell/rust seem better for strongly typed languages because of the compilation time information available vs go's type system