I have been dogfooding my literate programming tool wheel. So far I have been able to develop multiple webapps, a game and even a C++ based desktop app with it. I also think LP can be invaluable in managing your programming notes and system admin commands, not to mention writing html based books. You can checkout all the project links here. I dub my approach to LP as procedural literate programming. Instead of developing new syntax, I use string substitution based approaches common in web development which is more flexible than all the other approaches to literate programming so far, including the original cweb. I think the fundamental mistake that people seem to be making with LP is using it to describe algorithms rather than code history, code inspirations, bug stories, features and any other context surrounding code like tests / security. This means that git commit messages can be skipped with LP in some sense. While complex algorithms definitely can use lengthy descriptions, you can assume your reader is a programmer and just show code as is for everyday code. I suppose if you want to keep existing ways of working with code you can copy paste the diffs manually back into the literate source code, but a well integrated IDE could perhaps avoid this step and allow you to use both workflows simultaneously. Last year I participated in a few jams which allowed me to explore different approaches to LP. I have developed a comment based code extraction and animation approach to LP called Loom and I was able to develop a very basic IDE using PyQt. It turns out that LP has a lot in common with Xanadu and Web because fundamentally they all are hypertext systems. Talk about a wild coincidence! Like Xanadu, LP is heavily dependent on two-way links called transclusions. If anyone knows Ted Nelson here I would love to tell him about my findings about how a transclusion based browser can act as an IDE to LP. I also found that LP can act a glue tool for generative programming. You can create multiple DSLs and pipe them through external scripts, while presenting just the DSL to the reader. This is something I am exploring in a compiler project where I am trying to generate C code from YAML. As of now I think LP is a huge plus for solo developers juggling multiple projects. Reading code in an understandable order allows you to pick up code where you left off. No wonder Knuth was so productive with it. You can checkout demos for wheel, PyQt IDE and Loom on youtube. I have also created a screencast recently to showcase my workflow with sublime

I think the fundamental mistake that people seem to be making with LP is using it to describe algorithms rather than code history, code inspirations, bug stories, features and any other context surrounding code like tests / security.

This sounds a bit strange to me. The original goal of literate programming, as introduced by Donald Knuth, was to formulate algorithms in a pedagogical way. I wouldn't call that goal a mistake. Maybe literate programming is not the best way to achieve it, but then... what is? That said, I am perfectly fine with your approach, which is more in line with the various notebooks that have appeared over the last years. From my perspective as a long-time user of Emacs-plus-org-mode, I'd say you are doing org-mode with Web technology. But I think this approach misses something essential: the algorithms on which all the high-level stuff ultimately depends. I want those algorithms to be just as accessible to readers as the high-level outline. With notebooks and friends, they disappear in a black box instead.

I am super into LP too, I use Observablehq, and I love mixing in tweets into my notebooks. You can tell a whole story, so you can connect a LP story to internet media (or in my opinion you should do). I also dogfooding my experience here https://futureofcoding.slack.com/archives/CCL5VVBAN/p1647859090612199 I also think git is less needed in LP situation, you just need the rollback safety features, and merging for team workflow. Funny enough we discussed that organically last week!

@Tom Larkworthy Yes its about the story! Does Observablehq allow you to embed content from other blocks, like noweb ? Jupyternotebook can't really do that and it is just linear. It is crucial to have that so that you can explain code in any order. @Konrad Hinsen To take quick sort as an example, I would 1. show the video of quick sort first 2. show the unit test second 3. explain the history of the algorithm 4. explain any trivia 5. give a brief explanation of the algorithm along with common bugs 6. put up personal anecdotes 7. put the complete quick sort code rather than go into details like declarations, loops, recursion ... I find examples going into too much details while completely missing 1-6 In the LP wiki quick sort is described as

*<<public declarations>>=*
*void* quicksort(*void* * base, size_t num_elements, size_t element_size,
        *int* (*comparer)(*const* *void* *, *const* *void* *));
*<<quicksort>>=*
*void* quicksort(*void* * base, size_t num_elements, size_t element_size,
        *int* (*comparer)(*const* *void* *, *const* *void* *))
*{*
  <<quicksort declarations>>
 <<check termination condition>>
  <<select a pivot>>
  <<partition the array>>
  <<recursively sort each subpartition>>
*}*

I would basically avoid declaration chunks like the above snippet ... unless its really a complex algorithm. Most everyday code in apps is far too simple to warrant such detail.

yes, observable is dataflow driven and features non-linear (topological) execution order https://observablehq.com/@observablehq/observables-not-javascript

@xyzzy I think the challenge is to link all that together. Example: I have a notebook that calls a function called

quicksort

. As a reader, I want to figure out what it does. First step: look up the code that is actually called. That's your step 7, and should be trivial, which it is in a good IDE but (so far) it is not in notebook-like environments. The only other piece that a code analysis could lead to is unit tests (your step 2). Everything else would be annotations added by the developer, so now we are in the "documentation" category. Donald Knuth's prose gets complemented by other media, links to Wikipedia, etc. All the niceties that didn't exist at Knuth's time. But that's just a technology update, not a fundamental improvement compared to the LP of the past.

Roughly speaking what I propose is not an update on LP, anything that implements tangle and mangle is LP, but an update on the code documentation genre / style. The final LP document should read like a history book as opposed to technical description of the code. My goal is to capture the evolution of all technical decisions made in a project, algorithms unless they are critical ... like say complex signal processing are of secondary importance.

This one was a style I liked https://observablehq.com/@tomlarkworthy/mip. BIG intro to topic, executable tests, actual implementation was really done elsewhere. Sorta looks more like a text book in the end. I have a change log for the history, though I tend to be quite terse on on that bit. I don't think history is very interesting if you end up reading about doing something that 3 months later is deleted and replaced with a different way. I use history to answer... did anything change since I last looked? If you want an overview of how a whole community is using LP (which is different to how I do it) I have a 100 of the best here:- https://observablehq.com/@tomlarkworthy/notebooks2021

I like obervablehq's style! What a difference it makes to show examples and tests first. In many ways blogs are literate programs too. While a history approach may not seem useful for discussing math heavy topics, it becomes very useful for showcasing project evolution. Right now I am evaluating DuctapeJS and Lua. Instead of throwing the final code away, I will just add a section on scripting engine under early prototypes chapter for example. Let us say a project is changing from apache to Nginx or from REST to Graphql, all these can be discussed in github issues and then the code changes can be documented. Heck, you can even have a toggle button to show old code - as I am using web stuff for doing LP, this is entirely possible. I right now use a "DEV" flag to hide stripe API keys. Many decisions like framework choice, architecture choice, requirements can go into LP. Why are you building the damn thing in the first place ? When a function is deprecated I would like that to be documented too. Right now this is scattered in pm tools, github issues, mails, ad-hoc documents and commit messages. Apart from examples and tests, add a tutorial and user guide if you are building a reusable library. I am using wheel to build everyday polyglot webapps, where most decisions are actually related to UI, so I am also documenting colorschemes 🙂