Reverse projection as a way to do spatial computing. This is a demo of js-ar-toolkit5 connected to leaflet or tldraw that uses a camera to project stuff on smartphones. Inspired by dynamicland, but replacing the paper with phones and the projector by a camera. It makes the phone more a tool for creation instead of consumption and makes use of our ability to move (ourselves and our phones) around to create meaning. Curious to learn if there is other use cases where this could make sense; currently thinking more about brainstorms or museum exhibitions where visitors can contribute to the exhibition and what other experiments are done in this space. The source code can be found there https://github.com/lemio/ReverseProjection and can just be tested when ran on a laptop and a phone. The source code is very vibe coded; more to get a working result than to make it ‘good’. Multiple phones does not work reliably at the moment, but is on my todo list; other types of hardware like a bluetooth keyboard with some marker would also be cool; so it becomes easier to ‘type in space’.

Cool. It's like scrolling by moving your phone. Very cool idea! edit: maybe with IMU tracking, the camera could be made optional?

It could definitely support, but not sure if it can replace, since you lose some absolute positioning especially when doing it with multiple phones/people or when using the physical space as a reference (e.g. a printed map where the phones are placed on top of). But just for replacing relative scrolling it could indeed work.

its sort of like AR. Might be good for augmenting with extra information, like if it was over a map of the world you could put (live?!) stats on it or something. Would be good to make use of the other things a phone can do to add to an inanimate object.

I’m amused you say that the phone is the paper and the camera is the projector – I would have said that the phone is the projector (the means by which a physical surface is “illuminated” with virtual contributions) and the camera… well, the camera doesn’t really have an analogue in Dynamicland, cuz you’re only using the camera to track the phone, which only serves as a lens to a virtual projection on the table, and Dynamicland doesn’t need that tracking cuz it doesn’t need a lens. Another fun metaphor is to the use of fiducials to track VR headsets. I was going to say “HTC Vive!“, but actually, Vive tracking is _inverted_: the headset is the sensor and the lighthouse is the projector. Helmholtz reciprocity is a trip!

The map demonstration makes me think this could be a pretty fun interaction model for a game.

I was thinking about this concept too, and you can use the back camera of the smartphone to track your position in 3d by placing QR codes at the table or on the wall

@Oleksandr Kryvonos ow cool idea; that would even make it more flexible and passive. Curious if it would be accurate enough; but one way to find out. Curious if this would work without some ‘whiteboard calibration’ for the different type of smartphones. The QR code could also be part of some funky ceiling design.

also you don't need QR codes, just some doodling or in my case hand-writing 🙂 will be enough to work as an visual achor (or should be enough)

I was looking into this for the map example; but couldn't find good libraries to do this (especially in the browser) my thought was indeed that the map screenshot should be enough distinguishable so it wouldn't need at QR as fiducial marker.