A completely unrelated question: Every programming language has a notion of "dependencies" between data, e.g. the between the variables (or functions) of a program. Understanding this dependency graph is important: it helps you determine the flow of data through your program. Consequently, it's problematic if every value is accessible from everywhere. Programming languages usually solve this with a notion called scope. Scopes are typically nested. Scopes may take the form of "modules" or "blocks". It's my understanding that organising data into scopes is a mere approximation to expressing the true dependency graph of a program's data (which can be a DAG or a cyclic graph, not just a tree). It's easy to see the extent of a value hidden in a block, but it's not possible to show that a value is referenced in precisely two blocks. Similarly, it's easy to make values internal to a module, but PLs don't have a built-in way to visualise how the imports and exports of modules are interconnected (there are some external tools for visualising this). My question is: has anyone seen some good ways of representing data dependencies? Something better than a boxes-and-arrows visualisation?

Just design a language built around the idea of data dependencies. You're welcome.

Sure, but how does that language represent those data dependencies?

Well in Onex, the rules have a left-hand-side matching part that goes out and plucks data (dependencies) from the surrounding network of objects. You just look at the left-hand side.

is there a picture that can summarise what you're describing?

there's stuff on http://object.network but I may have to draw something extra for you..

It’s one of those things I’m confused why it isn’t a standard feature of IDEs

@Duncan Cragg So I looked at Object Network. Am I right in thinking that Version 1 will be an Android app that acts as an Object, and will access the devices functions, which can then be accessed via a HTTP/API call using the Object Link mechanism to control the phone remotely?

@Zubairq thanks for having a look at my project 😄 Version 1 will be the Mother of all Switch and Lamp Demos! Showing how you can turn on lights with buttons, etc. Yes, the "IDE" will be running on Android. Onex also runs on nRF5 devices such as the Puckjs: https://www.puck-js.com/, providing the button and light i/o. Each device is represented by its device object, that then links to multiple further i/o objects - so the Puck has a single device object linking to a button object and to a light object. Seeing those in the Android Onex app, you can wire the button over to the light so that it turns on when the button is pressed! 😮 😄 Inside the Android phone, I intend to access all the sensors and actuators, data and APIs gradually over time, adding to the linkable objects visible in the UI.

Ok very interesting what you describe but I think I will need to see it to understand it. I have seen hundreds of iot demos but they always are very techie and not useable by normal users

@Peter Abrahamsen Graphical notations form a language, and the programming environment I’m aiming for is not based on text files, so such notations can definitely be a language feature. The use case is tracing data dependencies across Prolog-like rules.

@Nick Smith looks like there are sone useful results for “provenance graph”. There might at lest be useful considerations for information design in the literature. But you’ve described a domain, not tasks or activities that might be aided by a notation. I don’t know what form your non-textual language takes, so I don’t know what might be relevant.

The primary task to be assisted is broad code comprehension - can I understand what the program is doing? How all the pieces fit together? All other tasks are subservient to that. I’m not targeting any further niche at the moment — I’m trying to put together a core syntax. The language is not non-textual, it has plenty of text. It’s just not limited to a file-sized Unicode text string. Most of the visual design is still on paper so I don’t have screenshots yet. I’m looking for inspiring examples of dependency represesentation, on the chance that there’s some existing work out there. So it doesn’t need to be based on any particular syntax I have planned out. Searching for “provenance graphs” pulls up cliche boxes-and-arrows which doesn’t really yield any insight.

@Nick Smith two approaches I've been exploring in my tree languages. 1) Cubes. This one is inspired from mahjong. In Tree Notation, a program is isomorphic to a spreadsheet. A newline separates a row; a space separates a column. Everything is typed and types can inherit from other types. So I have methods where you can visualize a "TypeCube" of every program. So you can see the type dependencies of each word.

@Nick Smith I like your thoughts on this. Let me add that having used environments with different scoping conventions, it turns out that the mess of having many symbols in scope tends to be less than the mess of threading data through to where it is needed but the promised land would be good, dynamic ways of understanding data dependencies.

@wtaysom Yeah I'd like to experiment with having the computer pull up the dependencies relevant to the current focus, rather than having the programmer organise them manually. The question is: what does that look like? I may have to just play around in the absence of prior work.

@Nick Smith when it comes to gathering relevant dependencies, a good trick is to look at the edit history: bits that were recently changed likely relevent as are bits that were previously changed around the same time as the focused bit. This is especially useful for long distance dependencies. Collapsing library/framework code is a special case of this principle.

@Nick Smith sorry, that was just one that I had open in a tab. I haven't shipped a good interactive version yet. But you are looking at the type stack for each word. So if the word in your program in some language was 3.14, the parser might assign it to the type π which extends from float which extends from numeric which extends from any. So in a type cube visualization it me be a stack of tiles with 3.14 on top, underneath would be a tile with π, then float, then numeric, then any. So the stack would be 5 tiles high. Now if your language didn't have the concept of numbers at all and perhaps was just strings, that stack would only be 2 high (3.14 and any). So the "height" is an approximation for the amount of meaning in a program.

If you’re curious about representing dependency graphs in general, the best survey paper I’ve seen is this one :

@Cameron Yick ooh that research seems interesting. I'm going to save that as a reference, thanks!