, by @Dave Ackley. The subtitle "

Bottom up engineering for robust-first computing▾

" is more informative. It's about building computing systems that remain robust under scale.

I believe that the biggest scale at which we should be deterministic is a single small object. And even that should not even be imperative determinisim, but declarative. Just a data point for the discussion!

What's a "small object" for you? A Raspberry Pi? A laptop? Something else? And your "should" comes from which perspective? Technical limitations? Human limitations? Something else?

😁 OK, I mean small data object like a contact, paragraph, sensor value, invoice line item, etc. That answer being about data not hardware may impact the list of challenges you would have followed up with, but to answer anyway: my "should" is just me being me, in the same way every "should" is based on personal perspective, and even feeling. And I think none of your "limitations" are involved!

I agree that probabilistic emergent computation has the potential to be more energy efficient and ultimately scale better (see the brain). However, its also much more difficult to debug and understand, so in a engineering sense it kinda scales worse because it does not really compose (or decompose) in an easy to grok way. It resists divide and conquer problem solving. Brains are crap calculators. More generally, I see too many people giving up on making things deterministic early. People forget the database scales incredibly well vertically and that thanks to transactions you can hide all the vulgarities of distributed computation behind a database channel. And thanks to Spanner and atomic clock, its even possible to scale it horizontally with a ton of money if you really need to serve everybody on the planet consistently with low latency. I think the reason we can build out crazy solutions like Spanner is because the deterministic engineering discipline can solve a narrow problem and then fit it into everything that needs it. Emergent computation is slower to evolve perhaps? Maybe an analogue is the written word, you can communicate globally knowledge orally via local-to-local, but its much better faster using libraries and the printing press coz they scale across time and space losslessly.

I agree that emergent computation is a different beast than traditional Turing-style computation. Much like quantum computing. None of these newcomers will fully replace what we have today. I expect us to end up with multiple coexisting computation paradigms.

neural architectures have random dropout in their training which makes them suitable to the probabilistic computational paradigm quite well.

Spanner

As a counterpoint, the cost of global strict serializability in spanner can be so high that you need caches and other denormalized stores for acceptable performance. Now, adding even one cache immediately defeats any consistency properties because (spanner + cache) does not have the same consistency as spanner alone. Also, you cant run analytics on spanner either so you need another copy in an analytics database. All these extensions introduce complexity into the system and need hand-written code to deal with consistency corner cases. In fact I think the dbs that "scale" are a great example of how strong determinism doesn't scale. What I think would work better is give up strict serializability across the system, but track the various inconsistencies. Maybe we can have a managed eventually consistent system (rather than an ad-hoc one). One way of doing this might be to allow different versions of the same object to exist in different parts of the system, but use version ids or logical timestamps to track the history and relation. Allow divergence where needed but use local rules to resolve them. This requires some core principles, like how to use logical timestamps across the entire system, but does not require the whole system itself to be serializable.