this was so refreshing to read: https://litestream.io/blog/why-i-built-litestream/

This echos my philosophy as well - almost no one is building a system that really requires more than one machine. The belief that we are has caused us to add so much incidental cruft that if you use industry best practices, you actually do need multiple machines. Taking an extreme example, the entire write load of twitter is only ~6k tweets/s, which can easily fit on a single machine. As another, there’s a great paper from the hyper folks that showed you could serve all of the queries to wikipedia from a single instance. If you use them well, computers are insanely fast. The problem is just that we use them really poorly and because of the incentives around software, we’re more likely to go with our first poor solution to a problem than taking the time to come to a good one. This is how we ended up with things like Hadoop which are millions of lines of code to run 100+ node clusters that can’t beat a laptop.

Yeah, the problem is that “fixing” this would require a whole lot of unlearning and re-education, which just isn’t likely to happen

TL;DR my take is that systems software engineering is just software engineering turned into risk management

My hope is to flip the problem on its head and show that part of the problem is we didn’t actually go high level enough. Languages like python still let you specify too many details and those details are what prevent us from creating compilers that output extremely high performance programs by default. We can’t expect everyone to magically become great systems programmers, but giving them tools where a single integer add takes ~30 instructions is hell of a handicap to start with.

Well put @ibdknox @Srini K. I think this is a general problem of software development, not just “systems engineering”. Too much focus on trends / hypes / frameworks, which causes overcomplicated solutions for even the simplest cases

Fun related anecdote, the, original NASDAQ system ran on a single machine (using Java no less)! https://signalsandthreads.com/multicast-and-the-markets/#2923 It actually made things a lot easier for making sure transactions were correctly synchronized. Then you just had to make sure the network routed things well.

@Yousef El-Dardiry yeah, I think the key really is having system level understand-ability. Even if a language was higher level but the abstractions it was built on are leaky / shaky, you’re in for trouble. Arguably Ruby / Python are higher level in some specific ways but systems understanding is poorer (hence the need for systems engineering / risk management / server-wrangling)

@ibdknox

My hope is to flip the problem on its head and show that part of the problem is we didn’t actually go high level enough. Languages like python still let you specify too many details and those details are what prevent us from creating compilers that output extremely high performance programs by default.

This reminds me of the Haskell ethos a bit? A language designed to describe pure computation at an extremely high level, then let the compiler figure out efficient ways to run those computations. The language is so high level that it didn't support the very concept of side effects (causing them to invent/discover the monad...) The result is that the GHC compiler truly is a marvel of modern engineering. But (devils advocate time), It took 30 years of the most advanced CS research to get here, and it still has extremely unpredictable performance gotchas in some places. Maybe it's because I've been jaded by our current world, but I can't see "higher level" as a solution. Mainly because "creating compilers that output extremely high performance programs by default" is actually a really hard problem for the general case. Part of the problem being: there is no single general case. @Srini K makes the point, a higher level language often results in poorer systems understanding, not greater. Or to put it another way, all abstractions are leaky eventually... How do you handle it when the leaks happen?

Haskell focused much more on things that are counter to what would make something understandable and fast, e.g. laziness.

First, I have to fan boy out a little. I followed Eve for a long time, and I'm a fan of your work. 😄 Thank you. It's an honor to argue on the internet with you 🙂

SIMD is actually a great example - it is exceptionally difficult to automatically vectorize because the semantics of most languages force your layout to be exactly as you described it.

I see where you are heading. Something similar already happened for register allocation when we moved up from assembly language. You don't layout your own registers anymore. The compiler has fancy graph coloring algorithms that do it for you.

fwiw, I think the DSL approach is totally legit too and I would argue that there’s a pretty thin line between DSL and well architected library. The main issue you face going down the stable of languages route is that eventually you want to take something from your nice sound DSL and use it in your robot movement DSL - then what? How do you translate semantics across different languages? And can people internalize all the little differences between the two? Something like LISP is pretty close to that and the conventional wisdom quickly became “don’t use macros (the DSL creation feature) unless you absolutely have to.”

That's a totally fair criticism against the DSL / zoo of languages approach. I don't have a good answer there either, just a personal preference 🙂 I will point out that the "smart compiler" approach has limits as well. This article is a bit specialized, but shows a practical example of automated where register allocation breaks down. Hardware engineers give preference to different registers for certain operations. It's hard to encode that into a modern compiler in an automatic way, though both Intel and ARM spend a lot of time and money attempting to do so. They often get "good enough" but it's never "optimal". Which is fine, but the problem is, because they are heuristic based, there is usually some input program that produces a terrible layout that completely destroys the performance, and then it's hard to fix without breaking other things. Computational performance is definitely a fascinating problem, with multiple approaches. 😄

Yeah, I think with something like layout it’s important that you can still manually specify it if you want. The problem with most automated approaches is that you can’t see into it or adjust it slightly without resorting to weird tricks like reordering completely unrelated things.

Computational performance is such a fascinating topic to me and it took me a long time to realize that it’s actually much simpler in concept than I thought. To go fast, you just can’t do much stuff. All of those super fancy structures we’d read about in research papers? They were almost always slower because in the end the only cleverness that matters is the bit where the total amount of work you do is less. You were often saving 10% here and hiding an extra 20% over there without realizing it.

This is a profound insight. I remember Jonthan Blow talking about just this in an older talk he gave: TL;DR lists are often fine! Start there, optimize only later / avoid premature optimization. All data structures are fundamentally about optimization. Fun story of him looking at DOOM source code when it came out how he was chewed out by John Romero when Blow said asset loading was done sub-optimally (back when he wasn’t enlightened!)

This is a really interesting topic. How does say Python fair here? It's the staple ingredient in most popular data science / machine learning oriented stacks. Most heavy computation is done indirectly via libraries who in turn call out to tightly optimized C++ or GPU code. On the plus side, that gives you the flexibility of a high level language to declaring your computational flow in with the heavy work done in optimized land. Of course, this also makes it a lot of "fun" if you want to customize / inject yourself into some part of the computation, since you need to drop down to Cython or so. On a secondary note, what do you think of Blow's Jai language? Here's a some information on Jai for the unfimilar - basically, C++ "but better", intended for games but quite general.

"almost no one is building a system that really requires more than one machine" — have hundreds of billions of dollars of transactions to prove it. Guess I'll retire after overseeing $1T. We've always used one server — running Rails! 🐺 Except for that one time when had a calculation that needed parallelizing over a few hundred EC2 instances. We used Redis.