Hello friendly internet people! I’m happy to share the current state of my tiny, incomplete, toy of an AR project for iOS / macOS designed to parse, analyze, trace, and render Swift application runtimes in 3D space. I’m posting the Github link here, and I thank anyone and everyone many times in advance for taking a curious moment to take a peek, and maybe even share your enthusiasm and interests! I am absolutely open to critiques, comments, questions, thoughts, and of course offers to work alongside you and your like-wise visualization/AR/VR obsessed team <3 https://github.com/tikimcfee/LookAtThat

Having tried it, what were your main impressions?

Well I’m mostly just happy to hear it ran on someone else’s machine, haha. I’m curious what you mean - what my impressions were of where I’ve managed to get so far?

Of the experience. What did it feel like to you to have your code out there in space? I have my own strong feelings, but I wanted to ask before sharing.

Oh! ❤️ The very first time I managed to get the code appearing on the phone, I had a strange overwhelming sense that I had suddenly gotten a picture of what I was actually writing. I had a sudden shift of “Oh, well I guess… that’s how much is actually there.” I started to recognize little patterns in the size and shape of what I wrote. When I loaded up other projects, I saw much wider files, deeper ‘stacks’, and without seeing the code itself, I felt like I had a sudden familiarity. Like walking into a room you’ve never been in before, but recognizing walls, furniture, interesting bits that seem visually distinct. Sorry if that’s a bit dramatic. I really enjoy this project, haha.

This looks really cool! I haven't run the project, can you explain me whats the code layout shown in this gif? How is it constructed?

@Daniel Garcia Hey there, I’m glad you like it! I can and I will happily. I even made another small demo gif. I’m going to try to put some more samples on the github page to help too, because it’s got a few more bells and whistles that kinda fit together. In that screenshot, the blue rectangles are directories. They are “stacked” by their depth in a directory tree, and every file is a simple long black rectangle rendered like a big sheet of paper. The trick, though, is that each glyph of text on the sheet is a 3d node - it can move and interact in space! In the gif, you’ll see I’m grabbing some directory out of a new rendered stack. Then, I turn on ‘hover info’, which shows the results of hit-testing the nodes underneath the mouse arrow. Finally, it does some syntax tree walking to figure out what the syntax/semantics are of that individual glyph, and all of its associated nodes. As I click in the AST node list, you can see the text that represents that piece of the syntax tree ‘highlighting’ and gaining some z-depth and color. And, as more nested stuff is selected, the nodes are free to move even further away to demonstrate that deeper nesting. That’s one of the first pieces of none-2d-only sample I’ve got. I’m not too far away from doing things like, “give me all the nodes for class declarations and make a grid out of them” or “stack up all the functions and order them by width then height”. This is where you come in - help me make more! 😉

Thanks for the detailed explanation @Ivan Lugo ! Do you have a roadmap for the features you would like to add to the project?

It’s a pleasure sir. However, I fear no roadmap that another human would find legible, heh. But now that you mention it, I think that’s the focus for today. I want to get a few more samples on the page, and a part of that will be figuring out where I am. I’m very curious if you have some ideas to bounce my way - what kinds of features would you like to see? (no pun intended. ok maybe a little.) Let’s say sky’s the limit - what do you wish you saw this doing?

@Ivan Lugo Thanks for sharing! It can be dramatic! I'm into visualizing execution traces. Imagine each step as a little wooden strip, a popsicle stick. Now you have a chain of them, a ribbon, millions of links long. Use rules to fold that ribbon. For example, here's a standard view that I like. Developed in stages: 1. Put each step below its predecessor. The ribbon goes from the sky to the underworld. 2. If a step is a call, put it to the right of its predecessor. Now, the ribbon zigs from left to right. 3. If a step is a return, put it to the left of its predecessor. Now, the ribbon zigs and zags. It's still quite tall. 4. If there's a gap between a step and the one visually above it, collapse it. Now this structure is compact with lots and lots of overlap in the middle. 5. If some steps are on top of each other, put earlier steps in front of later steps. Now we a nicely oriented 3D tree. Left to right shows call stack depth. Top to bottom is order of execution. Front to back shows things at the same depth with the stuff the executed first in front. Walking around the tree gives you some perspective. More comes from, say, making distances smaller the deeper you are in the stack. It still looks like a mess. But for me the real, real mind blower was to add a breeze. Suddenly the connections in this elaborate structure are as easy to see as the separate branches of a real tree.

@User I'm really chewing on this layout algorithm. I'm trying to think how I would implement this with my current structures. The library currently traces entry and exit of calls, so I could handle the left/right distinction. The list is a set of ids, so theoretically it's not too hard to pull from the list as many times as can be rendered. The sky's the limit indeed. This may sound simple, but I could render the persisted UUIDs that represent the call stack into a text layer. Then, stack these text-popsicle-sticks, and arrange them with the left/right pattern. The ifs have a static length so that's easy to make basic assumptions on for layout. It'll look like a big block of code, but indeed, with all those gaps and zig zagging, little mountains. I have a few questions about implementing that sort of packing algorithm! Haha. You mentioned introducing a breeze - I'm not sure about that one, is that a technique for some kind of graph interaction? This is a first for me all around for this tech and these ideas, so I have a ton to learn and absorb.

@wtaysom that sounds cool! Have you seen any programs that let you visualize execution traces?

I mean "breeze" as in "wind" as in.... like... pause right about here and it's hard from the mass of green where the edges of the trees are. Then when you play, between the breeze and the camera movement, it's a lot easier to see individual trees. Another thing is being able to shake a graph. Like, take this https://observablehq.com/@d3/force-directed-graph. Pulling nodes there is a little floppy. Add

simulation.tick(10);

to the bottom of the

dragged

function to stiffen it up. (Press the little play button to commit the change.) Then you can even tell the centrality of a node with the edges turned off

linkStrokeOpacity = 0.0

.

I have been calling the project that I'm working on a

Code explorer

. But I guess it could also be a tracer when I add live values to it (I hope not too far in the future 🤞). @wtaysom will love to hear your comments, if you have some time to spare and watch this video https://futureofcoding.slack.com/archives/CCL5VVBAN/p1650855082164669?thread_ts=1650340488.796839&amp;cid=CCL5VVBAN

19 seconds is certainly time I can spare @Daniel Garcia. I love it. Rewriting some code yesterday, I would really benefitted from a better browser. Let's see what I have open... here we go. An example: I want to show a thing on the Web using Rails. Exceptionally typical and illustrative. A templated snipped of HTML in `_current_assigning_products.html.erb`:

<td class='number'><%= format_total_blocks_to_assign_per_assignment_area_across_encumbrance_levels %></td>

We have a helper

clock_stage_results_helper.rb

because we need to insert slashes and commas between a bunch of numbers with "Block" or "Blocks" tagged on the end depending on plurality (and potentially language):

def format_total_blocks_to_assign_per_assignment_area_across_encumbrance_levels
    _f_aa(&:total_blocks_to_assign_across_encumbrance_levels)
  end

Now this private method

_f_aa

is going to iterate across assignment areas defined in

assignment_area.rb

calling

total_blocks_to_assign_across_encumbrance_levels

for it:

def total_blocks_to_assign_across_encumbrance_levels
    product_numbers.map do |pn|
      weighted(@clock_stage_result.winnings(pn))
    end
  end

So then the main branch here goes through

clock_stage_result.rb

basically looking up the

@sold_package

. It's a ClockStageResult::Package basically a product number indexed collection, basically a cache, a summary of a calculation. This is as far as a call stack is going to get you, but it totally makes sense to ask where does the

@sold_package

come from? What exactly is it summarizing about the clock stage? In layered code like this, the idea of having some blocks to show is split across many files: files for adding the presentational details, files for gathering together the total from the parts, the parts themselves, and where those parts come from. It's all well organized. Everything has and is in its proper place. I can even press ^z to jump from one definition to next, but I don't automatically have what I need right on the screen, and we totally could have the editor do this though it's not trivial. Two challenges come to mind: 1. The lesser: in Ruby, it's not statically perfectly clear from the text how the references work. For my editor, "Goto Definition" generally works, is sometimes ambiguous, "Goto Reference" doesn't work. I do a full text find. It's not terrible, it just slows you down. 2. Much more important: an editor needs some smarts to know what related parts to open up. In an scenario like this, there is a main path to follow (view-to-model, whole-to-part), but a system needs to be smarter than just opening up every definition. Consider this one:

def total_blocks_to_assign_across_encumbrance_levels
    product_numbers.map do |pn|
      weighted(@clock_stage_result.winnings(pn))
    end
  end

Of the things referenced here

product_numbers

,

map

,

pn

,

weighted

,

@clock_stage_result

,

winnings

, how do you tell that

winnings

likely has priority as the thing you want to inspect as opposed to

map

, which is the least likely?

This is a great example and so useful explanation @wtaysom. With your code I was thinking how tiny functions are sold as the holy grail, but our tools (especially editors) suck at displaying them in a useful layout. Follow up question: what are your thoughts about inlining code vs opening it in a new column at the right?

In Smalltalk, I would usually have a bunch of browsers open. Splitting panes in modern editors is awkward enough that I don't do it. Just minding some tabs: closing and reordering manually. In a way, I've no strong opinion on layout. I've seen inlining, opening right, and more. (When I do play with this, which is rarish, I start in 3D and imagine unopened paths as being in other dimensions mostly perpendicular, with just their entrance visible in projection.) My main concern is consistency. The worst offenders are mobile interfaces: a view that pushes in from the right and dismisses by sliding down. Loses all sense of space. Debuggers can be weird this way too. Consider a normal master/detail view. To make it really, honestly, spacial, a person should be able to, say, expand two details at once. Imagine flexible medium that you can stretch freely. Then in a context, there can (and should) be strong conventions about up/down left/right. What could we do with

total_blocks_to_assign_across_encumbrance_levels

? Let's say we inline local methods, stack loops (only usually showing a single focused example iteration but with borders to suggest how many times the loop is called), and expand calls to other objects to the right. That would probably feel good.