Hey guys, question for everyone who's making a desktop visual lang: what do you use for rendering and why?

Do you mean Qt vs GL vs DX vs GUI, etc.?

@Chris Maughan yep

My tool is designed as a shader authoring tool (in part), so my plan is to support rendering using GL, DX12, Vulkan. This is for the shaders/graphics the user creates. GL because it is a guaranteed lowest common denominator. DX12/Vulkan because they are the cutting edge. I don't want to use Metal if I can help it. For GUI elements I have 2 approaches: • ImGui. This sits on top of the API's I just mentioned and does buttons/checkboxes/windows, etc. • NanoVg. This is a vector drawing library along a similar vain to ImGui. It has slightly better support for pattern rendering, and is a bit simpler to use for basic primitives. It also uses GL, etc. for the bottom level of drawing. In this image, ImGui is drawing the left hand text editor, the sequencer along the bottom. NanoVg is drawing the Nodegraph UI inside the top right window. And everything sits on top of OpenGL in this example. My choices are based on ease of use and cross platform. Using custom GUI code for each platform is out of the question. I have a version of the text editor in the image that runs on Qt. Qt is a good cross platform choice for 2D/UI frameworks, but it is my least favourite API to work with (I do it for a day job too, so I am proficient at using it). Qt is powerful, but can be very frustrating. On the other hand, using it as a way to get a QPainter and draw shapes would be an easy cross platform method. I use SDL to do the heavy lifting of creating OS specific windows for drawing in. SDL is a great way to get a window on screen. ImGui integrates will with it. So my pathway is SDL->OpenGL/DX->ImGui OR NanoVg

@Chris Maughan Thank you! I already knew about imgui, my question was more targeted at rendering stuff like dataflow graphs, but nanovg might be what I'm looking for. I'll look into it:)

Something I am unable to use but would like to is the Slug library for text: https://sluglibrary.com/ As far as I can see this is the gold standard for rendering text on a 3D Device; it looks far better than what I have come up with so far (on a par with OS/GUI font rendering). Unfortunately it is commercial, though the author has hinted that he'd love some company to buy it and make it free.

Found this in the readme of nanovg, idk if it should but this kinda scare me away from using something...

This project is not actively maintained.

So you see, you can even draw triangles in ImGui if you want to.

Thank you!

I'm using NanoVG (which is OpenGL based) ... I know it's no longer actively supported, but it's in plain C and a lot simpler (IMHO) than imGUI.

I'd love to see more answers to this! I'm unhappy how state is managed in every OOP framework and every framework that uses a VDOM. I think neither mutability with objects nor mutability on another layer (vdom) works well. Building something else has been hard for me. Text rendering especially. Building something cross platform makes it 100x harder. Using the web platform doesn't work, when you try to build on a lower level: You can't even measure text dimensions easily. I've tried building up my own cross-platform solution based on WebGL, WASM (Rust), but failed at font layout. Compiling harfbuzz to wasm just didn't work for me, despite some people claiming to have done that. So, I'm considering non-web solutions again. Basically, I'm really interested what others have to say. Also, adding a reference to Raph Levien's work here, as I think he's also interested in this space: https://raphlinus.github.io/

@Philipp Krüger I'd actually be really interested in giving a rust-based solution a try :D

@Matei Adriel There's rust stuff happening here: https://github.com/linebender and also some relevant stuff (for me mostly) here: https://github.com/RazrFalcon/rustybuzz But it's very experimental. Hard to say what these projects are going to become, in what direction they'll evolve, and in what time they'd be viable. 🙂

Fonts are always hard, especially for smaller text. An HD display helps a lot - the problem is lessened on a 4K screen 🙂 As I said, the Slug library is probably an easy solution if you don't mind paying or just want a prototype. I've used Direct2D on windows before; that is nice renderer. Qt is very nice for fonts as long as you are happy to bite into that huge apple. Another approach is to use an OS specific renderer to draw your specific text onto a texture and then apply it in the engine; that gives you the best of both worlds, but you need to write custom bits. I've heard good things about HarfBuzz; I really should get around to building it.

I'm pretty sure harfbuzz itself is very mature, this is from their wiki:

The current HarfBuzz tree, used to be also known as harfbuzz-ng, is under active development and is what is directly used in Firefox, Chrome, LibreOffice, XeTeX, Minikin/Android/Flutter, PlayStation, Qt/KDE, GTK+/Gnome, Java, Emacs and Unreal Engine among many other places directly or indirectly.

Yeah, it's just big/complex.

WebKit direct DOM manipulation via Javascript

I was playing around with a shader editor and built on a similar stack as @Chris Maughan

I use Vulkan and imgui in my own project, Onex. And, to echo comments above, I also use lines and triangles in imgui to draw arrows. 😊 I had to remember some school maths. 😧

If I was working on a higher level tool I'd either do it with something browser based, or using a game engine

I tried browsers and game engines. Never again

why not?

Way way too much baggage. It's liberating starting from the metal (no pun intended)

fair enough, I think it depends on what you're working on

If you start with Sascha Willem's example code, it's pretty easy to get going

you can also go higher level than Vulkan + IMGUI and lower level than Unity or Unreal 🙂

OK. True nuff. Just wanted to be ahead of the curve 😊

if you're using subgroups in vulkan, or some fancy usage of descriptor sets or resource binding, etc. The thing about Vulkan is using it naively will give you worse performance in many cases than using GL or DX11 naively

+1 to imgui for rendering for something more complex like a fullblown editor, you'll need some statemanagement & databinding solution

@ikrima thanks for the link to lager btw, I wasn't aware of an Elm style library for C++

👍 the author's pretty great and I use his immutable persistent data structures library. honestly the biggest reason i had to roll my own was bc it's heavy template usage (and it links boost.hana)

yeah...

(more of a criticism of C++ vss. the lib; i'm not sure there's a way to do it in C++ otherwise. case in point, my version eschews type safety + uses C++ JIT codegen)

I already made a demo-ish visual fp lang using purescript (which runs in the browser) which was rendered to a html5 canvas

The thing about Vulkan is using it naively will give you worse performance in many cases than using GL or DX11 naively

Doesn't everyone use Vulkan naively? There are probably 3 people who actually know what they're doing, it's so insanely complicated. I rely on Sascha to be my code chef and sprinkle some of my own seasonings on!

Im comfotable in rust + a bunch of fp langs (f#, haskell, purescript, a few more)

I'd stick to C if you can, then. 😊

also just in case it was implied, i'd like to clarify that i do not recommend C++ but if you are forced in C++, here's some stuff. imgui i believe has a cimgui binding (used in sokol)

re: Godot: I went all-out with Godot but hit the issues I mentioned above

if Vulkan is working for you no need to switch

@Scott Anderson hot damn, thanks for that godot link!

Godot is the best OS game engine, don't get me wrong

For rendering my UI, what I want is a cross platform, high level, GPU-accelerated graphics UI that goes far beyond the postscript/SVG 2D graphics model. It should support hybrid 2D/3D, and it should be extensible using procedurally defined graphics primitives (signed distance functions) as seen on shadertoy.com. So there are no limits to what a UI can look like. You aren't boxed in by the SVG 2D rendering model or by what HTML/CSS can render. Shadertoy code has performance cliffs. To make this efficient, it should use an extended version of Raph Linus's GPU accelerated rendering architecture: https://raphlinus.github.io/rust/graphics/gpu/2020/06/13/fast-2d-rendering.html Briefly, there is a display list of primitives loaded on the GPU, the viewport is partitioned into tiles, and each tile renders only primitives visible in that tile. Cross platform means Linux, Windows, MacOS, Android, iOS and web via WASM and WebGPU. Anybody want to help build this?

@ikrima I'm using NanoVg for 2 reasons; firstly it is simpler than ImGui for rendering basic primitives, secondly it has better gradient pattern support (last time I looked); and I'm using that for subtle effects here and there. NanoVg needs only a few lines of code to draw stuff, ImGui needs significantly more for custom. Glad to hear you are using Zep; I haven't given it enough love lately; but I will shortly be doing another round of bug fixes. If you have a screenshot of it in-situ, would love to see.

So your guys recommendations are to learn more C?

There are language wrappers around all this stuff (maybe not much for shader code which is very c-like). So it's not essential; but I'd imagine a low level 3D coder would have an easier time with C/C++.

I personally use C++, since I'm doing low level coding, and all of the graphics libraries that I need to use are written in C++. There is also a graphics programming community using Rust, and they are doing some cool things, but it's tiny compared to C++.

If you're doing low level graphics programming C or C++, followed by Rust, followed by C# are probably your best options just in terms of ecosystems, platforms you can target, examples available, etc.

Hi! I've got an updated answer to an old, but interesting question: I'm thinking of running skia in the browser (that's the stack figma uses). It's quite accessible via their 'canvaskit': https://skia.org/user/modules/canvaskit

This looks great, very keen to stress test it

does it provide some extra functionality not exposed by browser's canvas? because afaik browsers use skia for the canvas implementation?

Yes! @Mariano Guerra . One critical thing that I've been missing is way more control over text rendering. You can measure the text size, find the position at which to place underlines, get the caret index for clicks, etc.

Actual text flow within a canvas-like API with real multilingual text shaping (using Harfbuzz) is a pretty big extra. It's also interesting that if one uses Skia in the browser, the API is then the same for native, allowing code reuse. Lastly, there's this quite cool idea to serialize Skia API calls on a remote server for display on a local machine (sort of like a vector version of what X's network transparency).