https://futureofcoding.org/ logo
Title
i

Ivan Lugo

05/04/2022, 3:53 AM
Hello friendly internet people! I’m happy to share the current state of my tiny, incomplete, toy of an AR project for iOS / macOS designed to parse, analyze, trace, and render Swift application runtimes in 3D space. I’m posting the Github link here, and I thank anyone and everyone many times in advance for taking a curious moment to take a peek, and maybe even share your enthusiasm and interests! I am absolutely open to critiques, comments, questions, thoughts, and of course offers to work alongside you and your like-wise visualization/AR/VR obsessed team <3 https://github.com/tikimcfee/LookAtThat
:amiga-tick: 1
👏 1
:thumbsup_all: 5
w

wtaysom

05/06/2022, 1:04 AM
Having tried it, what were your main impressions?
i

Ivan Lugo

05/06/2022, 1:06 AM
Well I’m mostly just happy to hear it ran on someone else’s machine, haha. I’m curious what you mean - what my impressions were of where I’ve managed to get so far?
w

wtaysom

05/06/2022, 1:07 AM
Of the experience. What did it feel like to you to have your code out there in space? I have my own strong feelings, but I wanted to ask before sharing.
i

Ivan Lugo

05/06/2022, 1:13 AM
Oh! ❤️ The very first time I managed to get the code appearing on the phone, I had a strange overwhelming sense that I had suddenly gotten a picture of what I was actually writing. I had a sudden shift of “Oh, well I guess… that’s how much is actually there.” I started to recognize little patterns in the size and shape of what I wrote. When I loaded up other projects, I saw much wider files, deeper ‘stacks’, and without seeing the code itself, I felt like I had a sudden familiarity. Like walking into a room you’ve never been in before, but recognizing walls, furniture, interesting bits that seem visually distinct. Sorry if that’s a bit dramatic. I really enjoy this project, haha.
👍 1
It’s goofy how excited I am to hear your experience, haha.
d

Daniel Garcia

05/07/2022, 11:38 AM
This looks really cool! I haven't run the project, can you explain me whats the code layout shown in this gif? How is it constructed?
Have you thought of how to represent code using 3 dimensions? I see in projects like this that code is always flat (2D) but we have 3 dimensions available!
🤣 1
i

Ivan Lugo

05/07/2022, 2:40 PM
@Daniel Garcia Hey there, I’m glad you like it! I can and I will happily. I even made another small demo gif. I’m going to try to put some more samples on the github page to help too, because it’s got a few more bells and whistles that kinda fit together. In that screenshot, the blue rectangles are directories. They are “stacked” by their depth in a directory tree, and every file is a simple long black rectangle rendered like a big sheet of paper. The trick, though, is that each glyph of text on the sheet is a 3d node - it can move and interact in space! In the gif, you’ll see I’m grabbing some directory out of a new rendered stack. Then, I turn on ‘hover info’, which shows the results of hit-testing the nodes underneath the mouse arrow. Finally, it does some syntax tree walking to figure out what the syntax/semantics are of that individual glyph, and all of its associated nodes. As I click in the AST node list, you can see the text that represents that piece of the syntax tree ‘highlighting’ and gaining some z-depth and color. And, as more nested stuff is selected, the nodes are free to move even further away to demonstrate that deeper nesting. That’s one of the first pieces of none-2d-only sample I’ve got. I’m not too far away from doing things like, “give me all the nodes for class declarations and make a grid out of them” or “stack up all the functions and order them by width then height”. This is where you come in - help me make more! 😉
And to add on because I’m a chatterbox, something on my radar is a “metaphor mode” where you can swap the rendering of each syntax item into a generic shape or back to text. Something like “classes are cubes, protocols are spheres, functions are pyramids” and then having them hierarchically organized in a similar manner to the directory. All ways to get “quick visual insight” into what’s currently rendered. Then, of course, there’s things like “draw a line through these functions to show their execution order” - I’m working on that with the execution trace now, and the flashing text is showing each traced call happening in order. Making a directed line from that dealing with loops and the like is a burning goal..
d

Daniel Garcia

05/07/2022, 3:37 PM
Thanks for the detailed explanation @Ivan Lugo ! Do you have a roadmap for the features you would like to add to the project?
i

Ivan Lugo

05/07/2022, 3:40 PM
It’s a pleasure sir. However, I fear no roadmap that another human would find legible, heh. But now that you mention it, I think that’s the focus for today. I want to get a few more samples on the page, and a part of that will be figuring out where I am. I’m very curious if you have some ideas to bounce my way - what kinds of features would you like to see? (no pun intended. ok maybe a little.) Let’s say sky’s the limit - what do you wish you saw this doing?
w

wtaysom

05/08/2022, 4:27 AM
@Ivan Lugo Thanks for sharing! It can be dramatic! I'm into visualizing execution traces. Imagine each step as a little wooden strip, a popsicle stick. Now you have a chain of them, a ribbon, millions of links long. Use rules to fold that ribbon. For example, here's a standard view that I like. Developed in stages: 1. Put each step below its predecessor. The ribbon goes from the sky to the underworld. 2. If a step is a call, put it to the right of its predecessor. Now, the ribbon zigs from left to right. 3. If a step is a return, put it to the left of its predecessor. Now, the ribbon zigs and zags. It's still quite tall. 4. If there's a gap between a step and the one visually above it, collapse it. Now this structure is compact with lots and lots of overlap in the middle. 5. If some steps are on top of each other, put earlier steps in front of later steps. Now we a nicely oriented 3D tree. Left to right shows call stack depth. Top to bottom is order of execution. Front to back shows things at the same depth with the stuff the executed first in front. Walking around the tree gives you some perspective. More comes from, say, making distances smaller the deeper you are in the stack. It still looks like a mess. But for me the real, real mind blower was to add a breeze. Suddenly the connections in this elaborate structure are as easy to see as the separate branches of a real tree.
i

Ivan Lugo

05/08/2022, 5:25 AM
@User I'm really chewing on this layout algorithm. I'm trying to think how I would implement this with my current structures. The library currently traces entry and exit of calls, so I could handle the left/right distinction. The list is a set of ids, so theoretically it's not too hard to pull from the list as many times as can be rendered. The sky's the limit indeed. This may sound simple, but I could render the persisted UUIDs that represent the call stack into a text layer. Then, stack these text-popsicle-sticks, and arrange them with the left/right pattern. The ifs have a static length so that's easy to make basic assumptions on for layout. It'll look like a big block of code, but indeed, with all those gaps and zig zagging, little mountains. I have a few questions about implementing that sort of packing algorithm! Haha. You mentioned introducing a breeze - I'm not sure about that one, is that a technique for some kind of graph interaction? This is a first for me all around for this tech and these ideas, so I have a ton to learn and absorb.
d

Daniel Garcia

05/08/2022, 1:35 PM
@wtaysom that sounds cool! Have you seen any programs that let you visualize execution traces?
w

wtaysom

05/10/2022, 3:17 AM
I mean "breeze" as in "wind" as in.... like... pause right about here

https://youtu.be/gdpJ-akDGdw?t=26

and it's hard from the mass of green where the edges of the trees are. Then when you play, between the breeze and the camera movement, it's a lot easier to see individual trees. Another thing is being able to shake a graph. Like, take this https://observablehq.com/@d3/force-directed-graph. Pulling nodes there is a little floppy. Add
simulation.tick(10);
to the bottom of the
dragged
function to stiffen it up. (Press the little play button to commit the change.) Then you can even tell the centrality of a node with the edges turned off
linkStrokeOpacity = 0.0
.
@Daniel Garcia I'd say Replay https://www.replay.io/ is the coolest tracer I've seen recently. It's use of "print statements" is very practical and well conceived. Usually you want very precise slices into the whole of the trace. Another view I really like, is to select, say, an instance variable then layout the trace so that every access of the ivar is on a line. So if you picture the whole trace as a ribbon, this basically makes a sort of cylinder of it looping back around every time you use the ivar. Tightness now corresponds to hotspots for that variable.
d

Daniel Garcia

05/14/2022, 10:33 PM
I have been calling the project that I'm working on a
Code explorer
. But I guess it could also be a tracer when I add live values to it (I hope not too far in the future 🤞). @wtaysom will love to hear your comments, if you have some time to spare and watch this video https://futureofcoding.slack.com/archives/CCL5VVBAN/p1650855082164669?thread_ts=1650340488.796839&amp;cid=CCL5VVBAN
w

wtaysom

05/16/2022, 3:23 AM
19 seconds is certainly time I can spare @Daniel Garcia. I love it. Rewriting some code yesterday, I would really benefitted from a better browser. Let's see what I have open... here we go. An example: I want to show a thing on the Web using Rails. Exceptionally typical and illustrative. A templated snipped of HTML in `_current_assigning_products.html.erb`:
<td class='number'><%= format_total_blocks_to_assign_per_assignment_area_across_encumbrance_levels %></td>
We have a helper
clock_stage_results_helper.rb
because we need to insert slashes and commas between a bunch of numbers with "Block" or "Blocks" tagged on the end depending on plurality (and potentially language):
def format_total_blocks_to_assign_per_assignment_area_across_encumbrance_levels
    _f_aa(&:total_blocks_to_assign_across_encumbrance_levels)
  end
Now this private method
_f_aa
is going to iterate across assignment areas defined in
assignment_area.rb
calling
total_blocks_to_assign_across_encumbrance_levels
for it:
def total_blocks_to_assign_across_encumbrance_levels
    product_numbers.map do |pn|
      weighted(@clock_stage_result.winnings(pn))
    end
  end
So then the main branch here goes through
clock_stage_result.rb
basically looking up the
@sold_package
. It's a ClockStageResult::Package basically a product number indexed collection, basically a cache, a summary of a calculation. This is as far as a call stack is going to get you, but it totally makes sense to ask where does the
@sold_package
come from? What exactly is it summarizing about the clock stage? In layered code like this, the idea of having some blocks to show is split across many files: files for adding the presentational details, files for gathering together the total from the parts, the parts themselves, and where those parts come from. It's all well organized. Everything has and is in its proper place. I can even press ^z to jump from one definition to next, but I don't automatically have what I need right on the screen, and we totally could have the editor do this though it's not trivial. Two challenges come to mind: 1. The lesser: in Ruby, it's not statically perfectly clear from the text how the references work. For my editor, "Goto Definition" generally works, is sometimes ambiguous, "Goto Reference" doesn't work. I do a full text find. It's not terrible, it just slows you down. 2. Much more important: an editor needs some smarts to know what related parts to open up. In an scenario like this, there is a main path to follow (view-to-model, whole-to-part), but a system needs to be smarter than just opening up every definition. Consider this one:
def total_blocks_to_assign_across_encumbrance_levels
    product_numbers.map do |pn|
      weighted(@clock_stage_result.winnings(pn))
    end
  end
Of the things referenced here
product_numbers
,
map
,
pn
,
weighted
,
@clock_stage_result
,
winnings
, how do you tell that
winnings
likely has priority as the thing you want to inspect as opposed to
map
, which is the least likely?
d

Daniel Garcia

05/16/2022, 4:57 AM
This is a great example and so useful explanation @wtaysom. With your code I was thinking how tiny functions are sold as the holy grail, but our tools (especially editors) suck at displaying them in a useful layout. Follow up question: what are your thoughts about inlining code vs opening it in a new column at the right?
w

wtaysom

05/16/2022, 7:52 AM
In Smalltalk, I would usually have a bunch of browsers open. Splitting panes in modern editors is awkward enough that I don't do it. Just minding some tabs: closing and reordering manually. In a way, I've no strong opinion on layout. I've seen inlining, opening right, and more. (When I do play with this, which is rarish, I start in 3D and imagine unopened paths as being in other dimensions mostly perpendicular, with just their entrance visible in projection.) My main concern is consistency. The worst offenders are mobile interfaces: a view that pushes in from the right and dismisses by sliding down. Loses all sense of space. Debuggers can be weird this way too. Consider a normal master/detail view. To make it really, honestly, spacial, a person should be able to, say, expand two details at once. Imagine flexible medium that you can stretch freely. Then in a context, there can (and should) be strong conventions about up/down left/right. What could we do with
total_blocks_to_assign_across_encumbrance_levels
? Let's say we inline local methods, stack loops (only usually showing a single focused example iteration but with borders to suggest how many times the loop is called), and expand calls to other objects to the right. That would probably feel good.
Data dependencies: when was
@clock_stage_result
set? Maybe "outline" data dependencies. I mean if I'm focused on
total_blocks_to_assign_across_encumbrance_levels
I show the code where
@clock_stage_result
gets set above the method. Since assignment is often written before use, it's likely to be higher up in the same file.