https://futureofcoding.org/ logo
Join Slack
Powered by
I've been having an extended, multimodal conversat...
# devlog-together
k
I've been having an extended, multimodal conversation with @Jack Rusher over the course of 2022, culminating in a couple of in-person conversations in recent weeks. Watching his Strange Loop talk (https://futureofcoding.slack.com/archives/CCL5VVBAN/p1665503282267399) finally got something to click in my head, and I put 70 lines of Lua together for this demo. I'm also including the .love file for anyone to try out, which is just a zip file containing said 70 lines of code. Instructions for running it: https://love2d.org/wiki/Getting_Started I can imagine driving this through Emacs or some other Slime-like interface (e.g. https://github.com/jpalardy/vim-slime)
Screencast from 10-11-2022 11 33 14 PM.webmrepl.love
j

https://media.giphy.com/media/jErnybNlfE1lm/giphy.gif▾

One of the big things about using a livecoding approach for GUI applications is that you can inspect and fix a problem in your running program while it is already in the faulty condition. The inverse of this is when you need ten clicks into some sub-menu to recreate the state every time you attempt a fix, which is both tedious and flow destroying. One guy who came up to me after my SL talk mentioned that while workin on a big commercial video game he had to play the game for 30 minutes each time to get back to the part he was trying to fix!
k
I've been suffering from Blub here. Me: you should take charge of your tools, automate whatever you can. Also me: bah, all this IDE stuff takes too long to set up, it's brittle, keeps breaking on me, I'm just gonna print, because it's reliable and I always have it. (Except wait, I can't even get print to work reliably anymore. That's good reason to shop around.)
w
I like to understand regular GUI interaction as taking a path through some state space. Then I can use all the equipment of state machines and parsers to represent (syntax) and analyze (semantics) behavior. For the pragmatics, the trick is to reify both state (what is here right now) and transition (the steps to get here) so that you can swap between the two: integrate steps to get state, derive steps from states. If it takes 30 minutes to get to the buggy state, then you should be able to get a pointer to that state. From there my thoughts revolve around making richly interactive systems tractable by explicitly separating more and less embellished versions and from managing the embellishment in a principled way. For example if you want to find bugs or at least glitches, try touching anything animated. The animation usually increases the number and variety of states in ways that are easy to overlook as a developer.
c
Now you just need to make Time a parameter, and you've got a live coding performance tool 🙂
w
Yes, exchanging time for space is a good way to think about things. What is a "script" if not a written version of something that usually happens over time? Steps in a procedure or lines of dialog.
j
https://pernos.co/ is perhaps relevant. Once I hit the screwed up state, I want to ask how I got here.
k
When I built this I wasn't sure LÖVE had a good way to recover from errors, but I think it can be done. I just need to call the event loop through an 
xpcall
that takes a second function for error recovery and returns first whether it hit an error. What should I do on error recovery? Answering that seems to require thinking about the representation of an app. Following various past demos in Mu and Teliva, I'm planning a repo of apps to consist of the following: • A numeric sequence of versions (analogous to git commits). • Each version updating a single top-level definition compared to its "parent" version • Each version pulling in other top-level definitions from its parent For example, suppose a version 
n
includes definitions 
foo
and 
bar
, with each definition coming from a unique version. • 
foo
was last defined in version 
n-1
• 
bar
was most recently defined in version 
n
. Now suppose we edit 
foo
. We get a version 
n+1
whose manifest looks like this: • 
foo
comes from version 
n+1
• 
bar
comes from version 
n
In this way, the complete manifest at some version might include multiple definitions. But only one of them was actually modified in that version compared to its parent. Ok, so back to errors. Say I'm at version 
n
and try to edit 
foo
. I submit 
n+1
in the UI. When my live program runs it crashes. The error recovery function simply switches the most recently modified definition (
foo
) from the most recent version (
n+1
) back to its version in the parent (
n-1
). When the next frame comes around, the UI continues to show version 
n+1
, augmenting it with the error message from the crash. Now I can try to make a fresh edit and resubmit, see if that fixes the crash. I think I can do all this in a few hundred LoC of Lua. I can get away with a simple implementation because a paradigm of GUI development (without threads) forces most functions to return quickly. Frames are a nice boundary for snapshotting, and state lives in some global location between functions, while work done with locals within functions can be thrown away after a crash.
j
@Kartik Agaram What if the visible bindings are a projection over a graph where every node is a versioned fn? (as opposed to versioning at the program level)
k
Yes, that feels like a more concise rephrasing of my comment. All the file fiddling could be seen as implementation details. Does that seem right? One difference potentially is that I'm imagining a single global version number rather than versions by binding. But that's probably also just an implementation detail.
Today I'm playing with two communicating LÖVE apps, a driver and the app being driven live[1]. Stdin/stdout doesn't work very well when it's a machine on the other end (and I want to support Windows). I spent some time trying out sockets, but that goes against the grain of Windows firewalling. Now I'm just using a file for communication. [1] After I'm done somebody else can implement the driver within Emacs 😄 I want some amount of structured editing that feels faster to prototype in LÖVE.
Some initial ideas of the protocol between driver and app: DEFINE name ... DELETE name GET name [n] (returns definition as of version n) MANIFEST [n] (names with active bindings at version n) TIMELINE [n] (returns say 10 most recent interactions before version n)
j
You might want 
COMPLETE symbol-fragment
for later editor integration (if you go the versioning route, need to specify that context as well)
k
Oh, you mean for typing in part of a name and autocompleting it in queries? 💡
Current state. The app started out on the right without a love.draw, and updated on the fly after I sent the code on the left. Now I can focus on implementing the protocol and the on-disk representation.
j
@Kartik Agaram Yeah, for this sort of thing in the editor:
k
Error recovery is looking extremely promising. I spent an hour live-coding with my kids and didn't have to pop a terminal once. I hope to do a video in a few hours. Lines of code needed to be live: • Driver: 118 (over a basic text editor) • App: 400 (just the framework bits before the app does anything)
Ok, here's a take I spent too much time on. It shows too little, takes too long, etc., etc. But it shows the error flow I have so far. It's clear much more is possible: providing commands to undo the previous edit, inspect variables, etc. The top-level options Common Lisp provides on an error. https://spectra.video/w/wkDB5fsjBNBbsqKXGhGzwT (video; 5 minutes) (Many thanks to @Ivan Reese for showing me how to finally up the volume on my mic.)
k
This looks very interesting, and promising. And got me interested in LÖVE. Is there any high-level overview of that framework? I'd like to know in particular what its limits are. You mentioned somewhere the requirement of short (in time) state update and drawing functions. That's good to have in anything interactive, so it's not much of an additional constraint. Any other important limitations? How much of the embedding system can LÖVE access? Files, apparently yes. Network, probably. But can I use the microphone of my Android phone, for example? It's not obvious to find answers to such questions in the various tutorials and reference manuals.
k
I find it's pretty full featured! Here's the microphone on Android, for example: https://love2d.org/wiki/love.audio.getRecordingDevices One limitation I've run into is lack of https support. That's coming in the next version, but could be 1-2 years away. On the other hand you can always add libraries for yourself from the larger Lua eco-system. Https comes from luasec. I've been ignoring that option in favor of a download-and-done cross-platform experience, but it's pretty well-trodden ground. Maybe even on Android 😄
k
Thanks for that summary and pointer! Integration of Lua libraries was another point I was wondering about. It's not obvious with a framework. Good to know it's an option, while not forgetting that this is the entrance to dependency hell.
j
I especially like the note about how little code it really takes to have a nice experience like this. 🙂
c
You have certainly excited me about the idea of using LÖVE as a prototyping platform! Love how simply you managed to put this together (a lesson I often need to relearn)
k
If any of you would like to try it out (say for some coroutine experiments, ahem @wtaysom), I've packaged up the two sides into a single repo at https://codeberg.org/akkartik/20221018-live.love
live-coding2.webm
w
Ha! You've tricked me. I'll have to give it a try.
Coroutines work! Super dumb example, text extracted from screenshot. Seemed better than sending an image:
Copy code
x=x+10

if x > 100 then
x= 10

end

silly = function()
return x
end

cosilly = coroutine.create(function()
local = 3
while true do
coroutine.yield(r)
coroutine.yield(r + 1)
coroutine.yield(r + 2)
coroutine.yield(r + 3)
coroutine.yield(r + 2)
coroutine.yield(r + 1)
r=r+0.1
if > 100 then
r=3
end
end
end)

on.draw = function()
color(1,1,1)
for i,c in ipairs(Circles) do
if i == 1 then
circ('fill', c.x, c.y, silly())
else
local status, radius = coroutine.resume(cosilly)
circ('fill', c.x, c.y, radius)
end
end
end
I put 
x
in with 
Circles
as a way to track syntax errors since pressing F4 fails silently in that case. As you know, runtime errors have a big red "stack traceback" popup that can over up what you were doing. I also learned that switching definitions and then trying to use the cursor keys, can break it with a blue screen:
Copy code
Error: text.lua:86: attempt to index local 'line_cache' (a nil value)
stack traceback:
	[love "boot.lua"]:345: in function '__index'
	text.lua:86: in function 'populate_screen_line_starting_pos'
	text.lua:583: in function 'pos_at_start_of_screen_line'
	text.lua:409: in function 'up'
	text.lua:318: in function 'keychord_pressed'
	edit.lua:293: in function 'keychord_pressed'
	keychord.lua:11: in function <keychord.lua:5>
	app.lua:34: in function <app.lua:25>
	[C]: in function 'xpcall'
k
Thank you for those reports! The text is on disk btw. Let me start printing out the location on startup. You can also select text in the driver with shift + arrow keys and press ctrl+c to copy to clipboard
I put 
x
in with 
Circles
as a way to track syntax errors since pressing F4 fails silently in that case.
I don't follow this. Did you mean you tried creating a new definition for 
x
and hit F4 and it failed without error? One weirdness with the current implementation: creating a new definition doesn't automatically update the buttons up top. You have to hit the 
manifest
button again.
I'm not able to get your code snippet running, so I'd love to get a snapshot of your save directory, maybe do a call with you about what you saw. However, I can confirm that coroutines seem to work:
Copy code
silly = coroutine.create(function()
	while true do
		coroutine.yield(love.math.random())
	end
end)
Copy code
on.draw = function()
	local _, r = coroutine.resume(silly)
	local _, g = coroutine.resume(silly)
	local _, b = coroutine.resume(silly)
	color(r,g,b)
	for _,c in ipairs(Circles) do
		circ('fill', c.x, c.y, c.radius)
	end
end
w
Yeah, why 
x
as a way to track syntax errors? It's so silly. I have the first circle use radius 
x
, which keeps changing so if I hit F4 and see a change to that circle I know the code parsed.
Open in Slack
PreviousNext
Powered by