Backend integration: Use linear or nearest neighbour filtering?

[realcundo]

Hi!

TLDR; When integrating egui with other backends, should I use linear or nearest neighbour filtering?

Long story: This is my first time using egui. I'm trying to integrate it with miniquad (which is also new to me). At the first glance it seems to work but the visuals are a bit off, some lines are more jagged than what I saw in the egui demo. I added a check whether the coordinate system is off by 0.5 and it seems to be the case. When using the following code

for i in 0..10 {
    let i = i as f32;
    let offset: f32 = i / 10. + 10. * i;

    ui.painter().line_segment(
        [
            egui::pos2(10., offset + 10.),
            egui::pos2(100., offset + 10.),
        ],
        egui::Stroke::new(1., egui::Color32::WHITE),
    );

    ui.painter().line_segment(
        [
            egui::pos2(200. + offset, 10.),
            egui::pos2(200. + offset, 100.),
        ],
        egui::Stroke::new(1., egui::Color32::WHITE),
    );
}

I can see the 5th line (counting from 0) to be the brightest one. When I copy&paste the same code into egui fractal clock demo, I see all lines having the same brightness which suggests nearest neigbour?

I'm using the latest git checkouts, one pixel per point. I've tried the latest and also more ancient versions of both egui and miniquad (from https://github.com/not-fl3/miniquad) and they all are off by 0.5 in my code.

Bonus question: when I draw something at (10, 10) -- is it the center of the pixel or its upper left corner?

[emilk]

Hello!

When integrating egui with other backends, should I use linear or nearest neighbour filtering?

Egui should work with either texture filtering mode. In particular, egui_glium and egui_web both use bilinear filtering, but sample the texels right in the middle anyway, so it doesn't matter.

If you are on a display with one physical pixel per point, then the default monospace font (at size 13) should look pixel-perfect with all pixels either perfectly on or off (even with bilinear sampling). Like this:

Magnified:

---

When painting anything but text, Egui doesn't use textures at all (it just samples the (0,0) texel which is pure white). In particular, the line_segment example you showed does not depend at all on any texture filtering. Egui anti-aliases lines, so it is no surprise that your lines will have uneven brightness. It also makes sense that i=5 is the brightest one, as the coordinates for that line will lie exactly in the center of the pixels (so it will be exactly one pixel wide, with no need for anti-aliasing).

There is currently no option to turn off anti-aliasing (oops!) unless you bypass Context::tessellate and instead call egui::paint::tessellator::tessellate_paint_commands manually.

---

when I draw something at (10, 10) -- is it the center of the pixel or its upper left corner?

Upper left corner. In particular, if you paint a rectangle that has min=(10,10) and max=(11,11), then that rectangle will be exactly one pixel wide. I added a test to verify this in e105719

On my 1 pixel_per_point display this produces pixel-perfect squares:

magnified:

This is the logical choice, as it means that a window that is 1000 pixels wide has coordinates ranging from 0 (left edge) to 1000 (right edge).

---

Depending on the rendering backend you may need to offset either the positions or the uvs by 0.5 pixels/texels.

Good luck!

[realcundo]

Thank you, the explanation and those reference screenshots really helped!

If I do the "monospace" test it looks the same (except I'm using Color32::WHITE). That said I'm still not 100% sure about proportional fonts, they look more blurry than I expected and if I didn't make the font white they'd be quite difficult to read:

Zoomed in:

It might be just me but I expected H or I to be pixel perfect. Also those CollapsingHeader triangles don't look right to me. In any case, I'll try more experiments tomorrow, at least I know for sure the monospace text is good!

Update: I played with 0.5 pixel offsets in vertex and/or uvs and it seems like the default offsets of 0 are the best. Perhaps the font size is too small to be clear and what I see is the expected result. Well, unless I'm doing something else wrong, which is a strong possibility. 😃

Update 2 (sorry!) I've tried to use eframe to output the same UI as above and fonts looks very similar so probably my assumptions about H and I were wrong. What is striking though is that visually the eframe version looks much neater, e.g. the header has a bar, when it's opened it has the vertical line. And I think the colour isn't even WHITE and yet everything is readable:

I think I might be trying to bite off more than I can chew! And maybe I should just switch to eframe and glium unless there's something glaringly obvious to you which my inexperienced eyes can't see.

Oops, Update3: I think I finally found the problem -- it's the shaders. When I tried to reuse the one you use in Glium I was gettign the results above. When I removed the sRGB code and implemented the fragment shared from the outdated miniquad example (RGB are all the same in the texture) the output looks right and text is readable even with the default gray font. Being the noob I am, what is the sRGBA calculation useful for?

[emilk]

Oops, Update3: I think I finally found the problem -- it's the shaders. When I tried to reuse the one you use in Glium I was gettign the results above. When I removed the sRGB code and implemented the fragment shared from the outdated miniquad example (RGB are all the same in the texture) the output looks right and text is readable even with the default gray font. Being the noob I am, what is the sRGBA calculation useful for?

I'm glad you found the issue! The sRGBA calculations are for converting colors in gamma space to linear space. This is a big topic: https://en.wikipedia.org/wiki/SRGB#The_sRGB_transfer_function_(%22gamma%22). The short of it is this: most often we store colors in "gamma space", which is a compact way of storing colors. But when working with colors (e.g. to multiply them), we want to first convert them to linear space.

In egui_glium I tell the renderer to do the gamma->linear transform in the texture sampler (so I don't have to). Glium also uses a linear-space framebuffer, so I don't need to convert back to gamma space before writing the colors again. I haven't looked at miniquad a lot, but https://docs.rs/miniquad/0.2.55/miniquad/graphics/enum.TextureFormat.html there seems to be no sRGBA gamma aware texture mode, so I'm guessing it's all in gamma space.

The linked miniquad shader code does all color operations in gamma space, which is wrong. This will be noticeably different from the (correct) egui_glium anytime we have tinted textures (e.g. non-white text). If miniquad doesn't support sRGBA gamma aware textures, then it is in the same boat as the WebGL renderer in egui_web, and the same gamma->linear and linear->gamma conversions should be used: https://github.com/emilk/egui/blob/master/egui_web/src/webgl.rs#L15-L81

[realcundo]

Thank you, that makes perfect sense! Originally I just wanted to add a simple UI on top of my toy project that nobody is going to use but it seems the integration turned out to be quite a journey! 😄

I've tried to reuse bits of webgl.rs you referenced, using srgba pixels and the same shaders but visually I'm back to where I was originally and I'm not sure why. I've seen your issue not-fl3/miniquad#166 and I think that for now I'll stick with the incorrect rendering which looks better than trying sRGBA. I'll try to tackle it at some point later on.

[emilk]

FYI: I'm working on an update to the miniquad integration in not-fl3/egui-miniquad#5