@Dennis Hansen Great workshop on Artifact. I recognized the situation where you have to synchronise updates to do their thing before letting the outer thing tick on. Its in this notebook which is also a dataflow graph based programming environment but that is not graphical https://observablehq.com/@tomlarkworthy/flow-queue It interesting seeing the same thing manifest. I think its because values propagate through the program asynchronously, but you sort of want functional reuse or recursion, but calling a function actually takes time now in the async world, so to get that "request/response" functional interface you need the converter from async to sync, which involves a synchronisation barrier to bring back the functional abstraction. After trying to work with it a while I sorta decided its annoying for some problems. The nice thing about a function call is the caller is stopped until the the call finishes, and the result is wired back to the caller, although many different places may make a call to the same function. Its harder to do on a static dataflow graph. I think the functional calling primitive is something you could probably put into to your cells as they have routing ability (remember who sent me a value and next time I get a value send it back to the initiator?). I share coz sometimes seeing the same thing in a different context can give helpful for perspective. See your work made me thing about the fact I put a queue in my flow-queue which makes it kinda of an actor too, even though the underlying dataflow paradigm of Observable does not have queueing between cells. I had to add the queue to get to the functional interface, to be able to process one thing at a time on the static dataflow graph 🤔

Awesome really appreciate the thinking. Interesting to hear about how the 'sync' behavior has manifested in your work - and honestly i've toyed around with the idea of doing sync at the cell level as well. The only thing that sort of bothers me is that then cells might have to be in this 'waiting for response' stage, which then the user has to be able to understand and control etc. But the reality is that its kinda the same with the current sync-by-op method, since those need to have the value in the queue waiting for the next run. Worth mentioning, once encapsulation exists, and assuming they are fully flexible, then technically i'll need to have them allow arbitrary waiting at arbitrary groups of ports so i once thats in i hypothetically could prototype a cell that works this way in artifact which would be cool. My frustration with the current sync-by-op method is i think summarized as this- the state of the cell must be fully processed by any single loop and reach the cell again before it can be processed again, and right now the state of the cell is sent along an waiting at a switch that decided the direction. Thats sort of the standard pattern. It feels weird that the resting state of a program has a message waiting in a queue. But that is sort of required for that op to be immediately reactive since that state has to propagate through a loop before returning, its almost like theres one canonical 'state' of the cell is being passed around and can only be in once place at a time, again which just makes it weird that the 'state' is just sitting as a message. I think doing something like you are talking about makes sense, the only weird thing about that is that now cells, in a way, have to also be switches. And have to track whether the state is 'here' or its in a loop elsewhere (which means new inputs should be blocked. Totally 'legal' in artifact world to add this behavior but i haven't arrived at something that feels quite good yet. But yeah tbh this is maybe the one thing i think about that makes me question the current architecture most so honestly great question/feedback. Also i'll add- cool that you added queues. Yeah when you have any waiting/sync it immediately pops out as necessary unless you have subsequent messages override previous ones which feels unholy lol. So yeah i've also un-intentionally arrived at the actor model too. And the once meta-programming is added then nodes will fit the definition almost perfectly which is strange and cool.

The unsynchronized temporarily confused values were what the (functional?) reactive programmers call "glitches". Some frameworks suffer from them (especially ones that "push" data as you do), some implement "glitch avoidance". In one directional DAGs, simplest fix is two-phased (1) propogate invalidations (2) pull from the end, recursively re-computing anything that was invalidated. This effectively does topological sort. But that makes control flow even more implicit and magic, dunno how to sanely mix it with deliberately cyclic, non-converging loops. What I loved about you going the opposite way, is how a single notation - arrows of several kinds - covers both data and control flow 👏 The "wait until all inputs arrive" mechanic looks like the Petri Nets formalism.

@Beni Cherniavsky-Paskin Thank you! Yep the tricky part is just finding rules that are understandable- but interesting to hear about the FRP manifestation of that problem- when trying to solve at first i did feel the pull to move into top sort territory. And yeah now that you mention Petri nets is probably the closest formalism- or furthermore discrete simulation

the other approach I am exploring is dataflow templating. I think you alluded to this too in your talk. Instead of invoking a shared dataflow subgraph with synchronisation at the entrance, instead, clone the subgraph and run it independently in parallel, this is a different way of reusing logic graphs. Shared nodes can fanout to all the copies for global modulation (e.g. configuration parameters). This replaces the queue with parallelism which is more Factorio-esq. The entrance barrier thing I think is a consequence of sequential execution thinking and maybe is not the ideal paradigm. I have not finished the templating notebook, I don't like the actual programmatic interface but that doesn't really matter, but in this example there is a common widget from drawing cloned to make a dashboard with shared time axis and individually configurable data dimensions. So it mixes both shared nodes and cloned nodes. https://observablehq.com/@tomlarkworthy/dataflow-template

@Tom Larkworthy very cool. I've been thinking about this approach a little as well - funny how this problem space we're exploring is really quite the same. I definitely think parallelizing anything that can be parallelized probably should be. So I tend to think parallelizing should be sort of built-in/ easy to do for encapsulated nodes- i’m still not sure exactly how that would play out, but the way I see it is really under the hood as you might have some meta programming nodes that duplicate the sub graph for any waiting inputs. And tear them down as they complete