Looking for terminology / what terms to google for: In some languages or libraries you can run a loop or closure over an array or list in parallel, just by specifying parallel execution (sure, some preconditions must me met for that to work). Some libraries have sophisticated ways of how to distribute execution over available cores, either with some heuristics or by giving you parameters to configure that. If I want to do that for hierarchical data structures, are there any well-known algorithms that help with distributing the load across cores based on the run-time structure of the tree data structure? I think map/reduce would be an example…? If so, are there other ones? Also interested (but not exclusively) in any connections to math. I think I'm talking about applicative functors here…? I'm basically looking for the parts of the algorithm that takes care of breaking the problem into parts and then later on combining it back together into a single result. Often you end up with a tree-like structure, but it comes from the algorithm, but what if the source data structure already has a hierarchical structure? waves hands frantically to compensate for insufficient description

Here's Odersky et al's paper on how they did it for Scala's parallel collections:

One key term is "work-stealing". https://en.wikipedia.org/wiki/Work_stealing Hah, and the Scala paper above mentions it 😄

Tucker Taft's Parasail language has a very clever automatic parallel loop construct. The whole language was designed from the start to make it easy and natural to do things in parallel. It looks like a normal language, but its automatica parallelization is so clever you barely notice it. And for those doing super big data, there is Cray's Chapel which allows you to not only spread workload across cores but also across machines, so a 2D matrix of computing engines can be harnesses. that is for those monster types of jobs that i know nothing about except that is the kind of tool one reaches for in exascale computing.

Not 100% sure what are you looking for @Stefan, but I would start by looking into how programming languages like Go and Erlang work internally. https://divan.dev/posts/go_concurrency_visualize/. Something more mathematical could be CSP https://en.wikipedia.org/wiki/Communicating_sequential_processes. And also I skimmed through this paper and seems interesting: https://www2.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-183.pdf