The graphics seem to be used in the author's band's albums: https://sixm.bandcamp.com. Very nice integration of all aspects of life 🙂

Also related but not as popular puzzles "If you only had a vertex id could you write a function to draw something interesting" https://vertexshaderart.com

And "If you only had time as input could you write a function to generate music" https://games.greggman.com/game/html5-bytebeat/

Also loosely related is the demoscene which doesn't have an official site but this one will do http://pouet.net They often have competitions for demos that take 4096 bytes or less (4k). You're generally expected to download and run the code locally and Windows is the most popular platform but there are youtube channels that have recorded the programs running, especially since many of them need a high end machine https://www.youtube.com/channel/UC96JVq-z0-0iHAkIkKp1_6w

Signed distance fields (and sphere tracing) are a really powerful technique for making procedural art using mathematics. The mandelbulb fractal that Kartik linked to is not something you can reasonably achieve using triangle meshes.

I haven't tried this yet, but I've discovered that you can go beyond standard shadertoy programming and employ all sorts of optimization techniques to make signed distance fields performant. For example, check out the game "Dreams" by Media Molecule, and this talk: https://www.mediamolecule.com/blog/article/siggraph_2015

One thing that makes SDFs fun is the mainstream attitude that they aren't practical. Which means there is more of an opportunity to push the envelope and discover new techniques, just because fewer people are working on this stuff. Curv isn't intended for things like GTA5, but is intended to provide a powerful set of primitives for creating 3D printable models and abstract, math-based art. The art you create won't look like the output of mesh-based modelling tools, which is actually a good thing. There are already lots of mesh based modelling tools, this will be different.

So @gman, I just looked at VertexShaderArt, but I haven't spent enough time to "crack the code". At first glance, all the demos look like things that could just as easily be done using shadertoy techniques. So what's the practical difference? What's easy with vertex shaders that is hard with fragment shaders?

There was a question on S.O. The person was asking how to get their 1000 circles using SDFs to render faster. SDF is basically

for each circle
   is this ray in the circle

That's O(N) per pixel. It gets a little better using vertex shaders as you just draw the circles so O(1) but the more complex the circle positions the more you end up wasting GPU re-computing on every vertex (or pixel in fragment shaders) something you really should have only computed once and passed in. Basically these shader puzzles are almost always big perf loss because of that. They are fun though

Dreams is amazing but it's NOT just generating a single shader with SDF functions for everything the user creates. In other words it's not making shadertoy shaders. Users create data, that data might be in the form of SDFs but those SDFs are used to generate more data that is passed to the shaders in an efficient format. The SDFs are not directly evaluated by the shaders like shadertoy SDFs.

SDFs provide an exact mathematical representation for a larger set of shapes than what can be represented using boundary rep. You can represent infinite shapes, and you can represent 3D fractals with infinite detail, and you can do deep zooms into fractals without generating and then storing quintillions of triangles in memory. SDFs support a rich set of operations, such as non-affine transformations (eg, bend and twist), blending, and morphing. CSG operations like union and intersection are fairly simple for the SDF representation, but are quite expensive and even computationally intractible for the triangle mesh representation.

Curv is a high level 2D and 3D modelling program that uses the SDF representation. The puzzle for me is, what's the best way of rendering a shape described by a Curv program? Since Curv is a custom DSL, I can write a custom optimizing compiler that transforms Curv programs into whatever representation is needed for optimal rendering.

So how do you union a thousand shapes using the union operator? One answer is: don't do that, use SDF repetition (or space folding) operators instead. A smart compiler could maybe do this transformation automatically, at least in some cases, but I haven't figured that out. Other answers involve using data structures. Accumulate the results of the union into a 3D texture, and then render the texture. Or, put all of the shapes into a bounding volume hierarchy (similar to a ray-tracing acceleration structure) and traverse that in the GPU. Or, convert the shape to a triangle mesh and render that. I didn't know about vertex shader hacking (thanks for the reference), so I need to think about how that could be used.

Curv is used for parametric design of procedurally generated shapes. The benefit of regenerating the shape in every frame is that you can hook up your numeric parameters to sliders and vary them in real time. This is very useful for interactively exploring a parameter space. If you have to build a data structure before you can display the shape, and if the data structure can't be built quickly, then you lose this capability.

I love the community around shader programming and the demoscene, which pioneered many of the concepts for high-performance graphics that are part of architectures and libraries today. What I love is that they are driven by constraints. In the past it was the limited capabilities of the devices, today it’s the limitations of the massively-parallel programming paradigm for GPU programming (or alternatively arbitrarily set limits on binary size or memory usage, etc.) that requires you to think differently and find novel solutions. Part of that is that the solutions are often not the most readable, and there is a drive towards overly complicate and impressive tricks that show off the programmer’s competence and cleverness to make something work that was probably deemed impossible. It’s about pushing the boundaries and going off the beaten path to find new ways of doing things. Occasionally that leads to some breakthroughs that are useful in other domains. In a way I think GPU programming gets easily overlooked when talking about the future of programming, but it’s going to be a very important part of it. Now that we discover more and more use cases for GPUs as they become more and more general computing devices, and even the smallest devices we carry around with us have powerful GPUs in them, the future will also be a lot more about distributing computation across multiple different computation devices, a few CPU cores, many GPU cores, cores optimized for machine learning, and a few highly-specialized algorithms implemented directly in hardware, e.g. for image manipulation, (de)compression, and cryptography. I worked with many game developers for who this is already a (sometimes painful) reality. It’s no longer that simple to say there’s graphics acceleration happening on the GPU and everything else on the CPU. As GPUs are a lot more programmable now, they can take over a lot more work, and the challenge becomes balancing out work done on CPU and GPU to deliver the best experience by utilizing and optimizing for the resources a specific architecture provides.

@Stefan said "This sounds a lot like composition over a monoidal structure is what you’re looking for." Yes. And @Sébastien recently reminded me of his need for a scene graph, which is different terminology for the same thing.