Question about how compilers work Is it possible ...
# thinking-together
e
Question about how compilers work Is it possible to generate example source code from an AST node and its corresponding lexer rules?
l
That's partly how some linters and transpilers work!
Eg. babel for js parses the modern js into ast, modifies the ast, and then writes back to compatible js source code.
e
oh, yes. that makes sense. can you think of a project whose source code is easier to study?
l
You can quite easily write your own babel plugins, that could eg. replace strings and numbers with random values.
e
I guess I’ll look at babel since you mentioned it
l
Let me know how it goes!
Would be neat to look through multiple asts containing eg. the same function, and extrapolating how you may use it from those examples, ie. similar to how some NLP algos relate words with nearby words, giving it the ability to generate sentences that sound sound, but with the added benefit of the ast; that the code will parse correctly.
e
yes, that would be nicer since, in that case, the code is actually more meaningful
l
Was recently thinking about something similar; how you could autogen UI from data, given meta info of eg. how often + in what context a user would like to access it, and partly what components goes together with each other and what kind of data.
e
the use case I have in mind is a quick reference for a language. this would serve as an easy reminder for the syntax of a specific AST node
l
Niiice! Context dependent learnxinyminutes.com!
Extremely useful to not have to visit eg http://fuckingblocksyntax.com every other minute XD Usually much easier to "get it" by looking at a bunch of closely similar examples, than reading long syntax definitions; at least initially to get going.
😄 2
e
yes. learnxinyminutes is a source of inspiration
btw, your idea about autogening UI also crossed my mind. to generate a UI that would make you enter a valid AST node
b
https://svelte.dev/ describes itself as "a compile step that happens when you build your app" ... and is all about generating UI code; so perhaps that's somewhat relevant to your generating a UI ideas?
l
It is :) Looking to go one step deeper with a custom realtime offline first decentralized graph database with binary data streaming using wasm with rust, builtin, created from a structural declarative language, supporting new UI paradigm(s) (infinite non-euclidean zoom + scroll canvas/space with fluid structure, vs the status quo of plain documents) (see webgpu), while being fully editable and having powerful action history for free, etc etc
e
I don’t know enough about svelte to tell I realize that what I’m thinking about is basically a modal editor. I’ll have to think if what I have in mind is actually practical
r
IDK if you are looking for the more "formal" side of this but, I'm reminded of these projects that I had at the bottom of my bookmarks: https://baturin.org/tools/bnfgen/ https://lcamtuf.coredump.cx/afl/ This kind of technique is used in fuzz testing quite a bit. If you think about formal FSM/Automata theory, the normal way we use a FSM is to feed a string and see if it gets the end state (that's what a parser does). You can also run a FSM in reverse to generate random strings from the grammar. The algorithm is usually something like: Start at the starting state, pick a random node, spit out the string that would match that node, pick the next random node, continue until you get to the finish state. Typically you have a way to backtrack out if you end up in an invalid node. Invalid nodes represent parse errors, so you would fail with an error if you were just reading some user input, but since you are generating your own string, the program can rewind out of the bad state.
e
thanks, Ray. I’ll look into them. I’ve heard of Automata Theory, but I don’t know more about it