I wonder if someone has experience in building user interface navigable without mouse (ex: on TV, game consoles, etc). I remember obsessing over the nav system because pretty much every UI system I worked with required wiring the direction keys manually, and making sure there were no "focus traps", etc. Anyway, this nice simulation of delaunay triangulation gave me the idea of a possible way to wire direction keys automatically. I think some heuristic like this could work: • Make sure every edge between to widgets connected through the euclidean minimum spanning tree is assigned to one of the cardinal directions. • For each widget, fill the rest of the cardinal directions using the relative neighborhood graph, and remembering which widget we "came from" in the opposite direction. As usual, the devil is in the details .... for instance: • I imagine each widget would have to be modeled with a bounding area instead of a single point ... • Care would need to be put if bounding areas overlap ◦ Some widgets may still require tracking its own focus before delegating to the pre-calculated target (for ex., I imagine a scrolling gallery in the middle of the screen surrounded with buttons... 😭) • Etc, etc, etc .... There's a reason most UI frameworks just ask the programmer to do wire navigation by hand 😄

Imagine it as a physics simulation where there is less resistance along the edges (eg. imagine doing an sdf to all the edges), and pressing up applies more and more up-force on the pointer until it snaps to a new closest node :)

You would maybe want to force the connections to T/B/L/R and using curves to connect the nodes. That should be both more aesthetically pleasing and give you a more obvious mapping to arrow keys in all cases.

@Leonard Pauli About applying force, I'm reminded of using arrow keys (just left and right) to navigate a continuum where fine tuning is important and you may want to jump quite far too. One relatively intuitive way is to have alternating left/right presses zero in and continued presses right/right/right increase the distance traveled.

It can also be nice to use modifier keys as magnifiers. (This is how I have configured me editing environment: right arrow moves one character to the right, alt-right moves by a semantic unit -- paragraph, sexp, or whatever, ctrl+right jumps more, ctrl+alt+right jumps more, &c.)

@Jack Rusher sounds like macOS (alt, ctrl) plus a bit (ctrl+alt).

@wtaysom Yeah! Also emacs style (C-b = back one character, M-b back one word, and so on).

Yes, of the Emacs bindings, I consistently use C-a and C-e.

UIKit’s Focus Engine is surprisingly straightforward and has been battle-tested by lots of developers on tvOS for a few years. Perhaps you can find some inspiration there? https://developer.apple.com/documentation/uikit/focus-based_navigation/about_focus_interactions_for_apple_tv