https://futureofcoding.org/ logo
#share-your-work
Title
# share-your-work
g

George Svarovsky

10/12/2023, 9:08 AM
m-ld is a component for sharing live information peer-to-peer. (Lots of materials at the link, and click the big green button to find code playgrounds.) It's intended to allow programs to be written "local-first", having a reactive local model that is concurrently editable from multiple remote locations, with a guarantee of eventual consistency. So, you more-or-less write your code as if the data is 100% local, embedding m-ld as a library, and magically find that other remote participants can read and write the data too (subject to security!). There are two main use-cases we're targeting – client apps, like office tools, supporting collaborative editing (GDocs-style); and microservices sharing information like configuration or master data. In both cases, you don't need a (extra) server for the data (though you do need a way to pass messages, for example a pubsub). I've been working on this for a while so there's a million things I could ramble on about, but I'm curious to know if the above pricks up anyone's ears. Do those use-cases sound, well, useful? Does my description make you want to know more, or, ho-hum? Any feedback is welcome; will return with recent focus later! Many thanks
d

Duncan Cragg

10/12/2023, 12:25 PM
CRDT-based? And how is it different from SyncThing (that kind of question people ask which is SOOO annoying cos it usually means they didn't bother to read or understand what your offering is!)
g

George Svarovsky

10/12/2023, 12:53 PM
Yes, CRDT-based. Heh no worries, all q's are good, shows me what I need to lead with in the docs 😀. So SyncThing is "not content-aware" – its atoms are opaque binary files. m-ld's atoms are numbers, strings, booleans etc. that live in a graph. (The graph could locally be represented as an object model, or structs with pointers, or whatever.) So what you're sharing is the application-specific data model, which might often correspond, when serialised, to the contents of a file.
d

Duncan Cragg

10/12/2023, 4:02 PM
Great answer, thanks, that's very clear. So on permissions: is there a primary owner that configures write-permitted collaborators per (fine-grained) graph chunk? And I'd guess a kinda separate read permissions for more people to see bigger chunks of your graph?
k

Konrad Hinsen

10/12/2023, 4:42 PM
This looks at the same time (1) very interesting and (2) scary. Very interesting for what it allows to do - I guess I don't need to expand on this. Scary because it encourages "apps" to enclose data in silos (the clones), which is the exact opposite of what I am hoping and advocating for. I do understand that apps can store the cloned data in persistent storage, such as a file system, in any format they like, and thereby make it accessible to other apps. But the focus of the documentation is on apps that "own" the user's data.
g

George Svarovsky

10/12/2023, 5:52 PM
> So on permissions: is there a primary owner that configures write-permitted collaborators per (fine-grained) graph chunk? And I'd guess a kinda separate read permissions for more people to see bigger chunks of your graph? There's a 'grand vision' flavour of answer and an implementation-specific one. To start with the latter, let me first say that m-ld chunks the world into 'domains', which are generally expected to be file-size things, which are cloned in-full. Each domain is its own graph, and the convergence guarantee covers the contents of that domain. Read permissions in practice (so far) have been nicely covered by just controlling who has access to the messaging channel for that domain. For example, using MQTT, the channel is a Topic, which can be access controlled. The baseline position for write permissions is, everyone has them – the component is intended for multi-editor writing. But that's not going to be satisfactory for every use-case. I did a whole research project on decentralised fine-grained write permissions. The long answer is here. In short, the domain starts with the first "genesis" clone, which establishes the security model and its local user, identified by public key, as the authority over everything. After that, permissions can change by an 'agreement', which is a data update that disallows any concurrent update and so can be used to establish a kind of chain of custody which can be independently verified at each node. At the moment this model is tricky to set up and not well enough characterised, so for now, fine-grained write permissions are definitely in the realm of the experimental. To return to the grand vision: we're pretty sure we can make domains less physically-bound, and be able to have big domains which are cloned only in part, as required. That would first require some refinements to the 'agreements' model. But also, these ideas go hand-in-hand with having untrusted nodes, which is the tricky bug-bear of any attempt to decentralise authority. (Sorry about the hand-waving. More thoughts on this are in the long answer linked above, in the 'trust' section at the end.)
But the focus of the documentation is on apps that "own" the user's data.
Ah! That's great feedback, thanks, I definitely need to address that focus! So, we're hoping the design of m-ld will actually encourage apps not to be parochial about their data structures. It's based on the W3C's Resource Description Framework (RDF). So it buys into a world of well-defined tools and techniques for data, including the ability to declare the data model in the data itself. I'm hoping that will encourage the use of standard and shared vocabularies. This is the same fundamental choice as in the Solid Project, if you're familiar with that. As a component rather than a platform, we don't have the same opportunity to transform data ownership like they do. (It'd be great to explore how Solid and m-ld work together though.)
k

Konrad Hinsen

10/12/2023, 7:10 PM
Thanks @George Svarovsky, that sounds a lot better than my first impression! A combination of m-ld with Solid sounds very interesting. The pod would provide the common reference that multiple apps can refer to.