There's the question of content (what should be written, what aspects covered). But there's also the problem of the representation medium - most documentation today is off to the side, completely disconnected from the artifacts it is talking about. This is a fundamentally fractured model. You not only have out of date stuff, but constantly jump back and forth. Instead of a clear distinction between 'docs' and 'code', ideally there would be a hypermedium with various views at different levels of detail: high level views (system design, narratives) all the way to low level views (code) - all of them interconnected and traversible.

I agree that code and documentation should be as close together as possible. When talking about different views, projectional editors come to mind. The data structure they use internally is more like an AST and therefore more expressive than the "source text" commonly used today. You could add a "comment" attribute to each node type (module, class, even parts of an expression) or introduce additional node types only suited for documentation (tables, graphs, ...).

The 'syntax' is important only because the def facto medium is directly edited plain text files. To get all the good things you have to give that up, which unfortunately leaves you in a bubble.

yeah, moving away from text you lose a ton of nice properties. I’m thinking searchability, easy generation / transformation, copy/paste.

All of those things get better, no? It's just your text oriented operating systems and tooling don't work for you anymore.

right, thats 40+ years of accumulated tooling

Yes.. and do we want the next 40 years to be different, or the same? 😉

haha, for sure. I don’t know specifically what you’re thinking of, I’ve just read other kinda of proposals for new program authoring environments that don’t take into account program comprehension. Which I get concerned about.

TBH, I'm not bringing solutions, only problems. But one creeping realization I have is that fixing the problem might mean first changing some of the underlying assumptions and entrenched realities.

'Comprehension' is key terminology here. Most people focus on 'readability' in the small rather than the sorts of global activities programmers need to do that look a lot more like reading comprehension. I tend to think text is a perfectly fine way to do comprehension. We just need to do a better job (in some fairly straight-forward, actionable ways) within the context of existing text-based systems. Maybe going beyond text will also be helpful, but that seems an open question. For now, text is an assumption I'm content to retain. @shalabh and I have argued this before 🙂

pulling out that distinction between readability and comprehension is useful

Great point. I'm careful now to use the phrase 'plain text files'. Things like spreadsheets, Subtext and infra-structure.org are excluded by text oriented systems like Git. You still have text in these systems, mainly at the boundary with the reader (not necessarily as a large blob though, but organized within a different structure). If you have projectional interfaces, much of the content could be text. But why version and store large blobs of text when you really want history and different projections of the richer interconnected structures that are being represented? That's my thinking anyway.

Ah, that's a good distinction that you've had to remind me of a few times now. Very subtle and seems to easily fall off my head.