I am back on the original goal of exporting my Obs...
# devlog-together
t
I am back on the original goal of exporting my Observable notebooks to my offline-first format (lopecode), in a way that preserves editability. I started a notebook called jumpgate, that can do the conversion, and then opens a PR against Github. The first notebook I am trying to get this working with is Jumpgate itself. I use lightning-fs + isomorphic-git to do the Github part within a browser. The prototype is working sourcecode on Observable: https://observablehq.com/@tomlarkworthy/jumpgate git PR opened by the notebook: https://github.com/tomlarkworthy/lopecode/pull/4 as my repo is configured as a static host as well, we can see the result of jumpgate after going through the jumpgate here: https://tomlarkworthy.github.io/lopecode/notebooks/@tomlarkworthy_jumpgate.html#view=S100%28%40tomlarkworthy%2Fmodule-selection%29 Currently its slow as hell as the git checkout takes a long time. lightning-fs is storing itself in indexdb so there is not need to do a full checkout everytime, we should be able to merge with what exists. I went with simplicity for now as the stateful logic gets complicated as you have to do different git commands depending on whether you have done an initial clone etc. I am not sure how to manage the PR branch either, I keep force pushing a single branch but thats probably not correct either, I should use a fresh one each time but then you need to be careful not to fill indexdb with orphaned work. If anyone has a strong opinion on what the ideal github workflow for syncing offline files is LMK The jumpgate does not add the extra functionality to make a notebook editable, I need to next mixin with my editor notebooks to get the thing to do what I want. Good stage though, Having notebooks open PRs is cool.
I am writing a computational blog post and I want the windows arranged a certain way. Jumpgate supports injecting the window configuration but I have to keep specifying it and its annoying when iterating. So I added form population via URL parameters so now I just need a web link to have a specific export preconfigured. That is the next step in exporting everything repeatably, by building up a list of exports as a list of links to Jumpgate. The other thing heavier usage of Jumpgate showed was the git checkout step was slow, over 30 seconds, bottlenecked by the necessary git CORS proxy, so I made my own isomorphic-git proxy. Its super easy to setup on CloudFlare and in reduced my checkout to 3s! The instructions are making their way upstream if browser interaction with git interests you. Still, the git checkout step is not scaling well. Every notebook is 1-2Mb, and they all get downloaded before I can push a new version. I experimented with merging empty repositories, that works for unseen files, but not for updates to existing files (merge conflict between unrelated histories). git sparse checkout should be the answer, but not supported by isomorphic git ;'( Other ideas welcome. There have been several failed attempts to add sparse checkouts to isomorphic git, so its not an easy feature.