FWIW: this is a reply I made elsewhere. Hoping it might be of interest here.

Thanks for sharing! @guitarvydas

UNIX pipelines were headed in the right direction, but, the shell pipeline concept has lagged behind advances in FP - and, pipelines were conflated with function calls, which ain't the same thing.

Concerning those two points, I'm really curious to hear how that's the case. In my mental model, pipelines are homomorphic to chained functions. I'd love to know how you think that the concept of pipelines could be expanded, or where exactly it is lacking.

Pipelines are not homomorphic to chained functions. I will endeavour to expand on this...

Thanks for the article!

• The key insight is that pipelines are about modelling data flow, while function chains are about control flow - they're different abstractions serving different purposes.

I see function chains as about both data and control flow. In plain language: I do X and get Y; then I give Y to Z and Z does its thing.

• A pipeline only passes data, not control flow.

It's true that pipelines are more "continuous", in that the next utility might start working on its input before the previous one finishes generating the output. That's a remarkable difference. But for most practical purposes, I still see them as sequential.

• Pipeline nodes do NOT determine which code runs next [pt: whereas functions do]

But neither do the chained functions, right? This decision belongs to the caller, the one chaining the functions in the first place. Overall, I'm not trying to win an argument, just trying to see what I'm missing. Thanks again for sharing your insight!

I do X and get Y

With a function, yes. With a node in a pipeline, it's async, hence, you

do X

and don't need to wait around. The callee just sends

Y

further down the pipe without bothering to return it to the caller. Once you break out of the habit of thinking only in terms of functions, other interesting combinations appear beyond just simple, sequential pipelines. [I can discuss further, but, am cutting myself off to KISS]

But neither do the chained functions, right?

No. A call passes

control flow

immediately to the callee. A function specifies the callee directly, hence, determines which function will run next. A node in a pipeline cannot do this. This is subtle, but, important. In a pipeline node, the node does not

call

any other node, it simply leaves a result in a queue, to be be dispersed later by a higher power (the "dispatcher" in UNIX, "Choreographer Parts" in my stuff </KISS>). Dependency injection looks like it sends, but only adds a level of indirection to the naming process, while continuing to choose who gets to run next. Calling and sending deliver data differently. Imagine a queue - call puts the data at the front of the queue, whereas send puts data at the end of the queue.

Thank you for the clarification! I think I get it now. However, if a function returns, isn't it the caller that determines who to call next? My homomorphism is roughly as follows: • Utility in a pipeline: receives inputs (all at once or streamed), produces output (all at once or streamed), doesn't know where the input came from, doesn't know where it goes to • Function call: receives input all at once, returns data all at once, doesn't know where the data or control goes to (unless you passed a callback to it)

Your questions and observations are wonderful! Thanks! These things are definitely different, and, the difference results in bugs and hard-to-reason-about-ness in async programs.

Function call: receives input all at once, returns data all at once, doesn't know where the data or control goes to...

The word

return

is loaded. It surreptitiously contains a scheduling decision AND it surreptitiously contains a routing decision (the callee must send data back to the caller - no choice. The decisions are hard-baked into the syntax. ... reaching for another way to say this (I seem to be batting 0 everywhere I try to expand on this) ... Syntax such as f(g(h(x))) does not allow for the possibility that j(k(h(x))) gets to run until f(g(h(x))) is finished, i.e. f(...) makes a data-flow decision (stack-based data transfer instead of queue-based data transfer) - AND - it makes a scheduling design - AND - f(...) blocks any other input from reaching h(x) until it decides to allow this. Let's say you press a key on the keyboard, then click the left mouse button. That's different than a click on the left mouse button followed by a keypress. You can't differentiate these with f(g(h(x))) syntax followed by j(k(h(x))) syntax. F(g(h(x))) is fine if you have a single thread and no feedback. That's the way computers used to be in 1950. Today, computers are no longer like that (internet, robotics, IoT, etc), hence, strict sequential syntax cannot cope with reality any more. The best you can do is to write two async programlets that describe the innards of these things, but, it is hard to write a 3rd program that connects the two (without a lot of futzing and edge-cases and gotchas). Pure functions are nice if all of your problems can be solved in a sequential manner, but, that ain't the case any more.

I think I get what you’re saying, Paul. In f(g(x)), g knows that f must be paused and that it will resume iff it relinquishes control. That makes sense. But am I right that that only becomes relevant when you permit a system description that incorporates more than just the function call or pipeline? If you write g in a f|g pipeline with the assumption that f has rescinded control for the duration as if it were a function call, then you won’t observe anything to contradict that notion unless there are also other data- or control-flow mechanisms in play, like g sends information backward to f, or both f and g access some external system like a wall clock. Like Federico said, they’re conceptually the same… until you complicate them, and I think your point is that pipelines are amenable to “interesting combinations” that function calls are not.

Thanks! Yes, I think that your description is clearer than mine, but, there are several loaded phrases and words, like "g knows that f must be paused", "assumption", "more complicated"... https://open.substack.com/pub/programmingsimplicity/p/simplicity-vs-functional-programming?r=1egdky&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

@guitarvydas two points. First, apologies it took me so long to respond, I missed your last message. Second, your last example made it click for me. If you have a user clicking the mouse and then 100ms later entering a key, those actions are not necessarily sequential. Very much indeed! The way I see it, each of those two user interactions start a new sequence. Those sequences are being executed at the same time (whether in parallel with many CPUs or concurrently by one, it doesn't matter). You might have to make them join into one, or keep them running separately. Indeed, the entire system cannot be modelled as one sequence. But I do believe it's possible to model it with parallel/concurrent sequences. I also envision being able to have a command "fork" that runs things concurrently or in parallel, and then optionally waits for all of those sequences to finish so that they basically merge onto a single sequence afterwards. The core assumption at the base of this view is that everything is decomposable into sequences that pass data. Control passing is not necessary. But what does matter is that whoever has to wait (for either the entire previous thing or a chunk of data) will wait. And I think that that's not only reasonable, but necessary. Thanks for the example with the user interaction, it really made it click for me!