Please help test smooth window resizing logic

[raphlinus]

As I've described in #17, I'm trying to figure out how to get good performance on resizing and also scrolling. It turns out to be quite tricky. I have a couple of approaches, and I'm trying to collect data on how close either is to being acceptable. Please test on your hardware and respond as a comment.

All testers

Let me know OS version, graphics card, monitor size, DPI scaling. Please test master (or #18, if that hasn't been merged yet). The test procedure is:

git clone https://github.com/google/xi-win
cd xi-win/xi-win-shell
cargo run --example perftest --release

Does the spinner spin smoothly? Does the frame interval hold steady at 16-17ms? Does it still perform well if the window is nearly fullscreen?

Grab the left side of the window and resize. Does the window frame track the mouse? Is it smooth or is there jank? Does the diagonal line stay glued to the upper right corner or does it judder?

Don't worry too much about the spinning when resizing - for some reason it can get stuck when starting a move gesture. I also don't expect the frame interval to be consistent while resizing, though smoother is better.

Windows 8.1 and higher

Also try the flip_hwnd branch. Master just uses hwnd render targets, but flip_hwnd uses a more sophisticated approach. In steady state, it uses DXGI swap chains (in the FLIP_SEQUENTIAL swap effect). However, flip presentation is not synchronized with window resize, so it switches to hwnd when resizing, then back to flip.

Do git checkout flip_hwnd then the same tests as above. Is there flashing or any artifact when entering and exiting sizing? Is performance roughly comparable?

Multiple monitors / multiple GPUs

My main laptop is a Gigabyte Aero 14, which has integrated HD 630 and a GTX 1060 in an Optimus configuration. The laptop display is connected to integrated graphics, and the external monitor to discrete. It matters which GPU is selected (which is generally the "make this my main display" in display settings); performance is always degraded when crossing between the two.

Based on testing, I think I always want to draw with discrete graphics, at least in flip mode (in hwnd mode I don't seem to have the choice). How is performance affected when "make this my main display" is selected?

Higher frame rate monitors are also trickier? Does the frame interval hold steady at the faster rate?

Thanks in advance to the testers!

[DanielKeep]

Windows 7, nVidia GTX 1070, 1080p, no dpi scaling. Edit: also might be handy to know: i7-2600K (at stock speed).

Frame times appear to be consistently in the range of 16.0ms-17.5ms. It rapidly flips back and forth. There's definitely a 0.5ms in there, but I'm not sure if it's 16.5 or 17.5. The spinner appears to turn smoothly. The situation is unchanged with a maximised window.

One oddity I noticed when maximised: the window appears to be inset by one or two pixels on the left, right, and bottom. What I mean by that is that I can see the non-client window border around the edge of the screen. I have to move the cursor in by two pixels to get the "text selection" cursor to appear. Checking with an Explorer window confirms this is not normal. Clicking on this border doesn't appear to do anything save focus the window, and the window buttons indicate the window is maximised, so... I dunno. 's weird.

Resizing from the left toward the right is mostly smooth; the diagonal line will sometimes "jump". It appears to be the top-right vertex moving to the right, outside the bounds of the client area. However, this glitch is inconsistent and doesn't appear on every frame.

Resizing from the left toward the left leaves behind a white gap on the left of the window. It also exhibits the same inconsistent jumping of the diagonal line, except that now it jumps to the left, and leaves a horizontal line segment from the jumped-to position, across to the actual top-right corner of the window. Looking at the text, it doesn't appear to be smearing (i.e. stretching the right-most column of pixels from the existing bitmap), but I'm not sure. It's hard to tell at 60Hz.

Also, to clarify: the white gap on the left appears to consistently be the difference in the client window rect between frames. The diagonal line does not appear to have the same behaviour.

I can try to record a video of this in action if that will help.

[LPGhatguy]

Windows 10, GTX 1080, 1440p @ 144Hz, no DPI scaling. Commit 8e306fd.

Frame time alters between 6.0ms and 7.0ms, which is right on target for my display.

Spinner runs smoothly, unchanged at full screen. It is using 3.5% of my 8C/16T CPU, which seems a little excessive, reasonable if you're invalidating a lot of the window.

Dragging the left side of the window feels perfect, with no stutter or desync from my mouse cursor.

flip_hwnd branch seems to work the same. I didn't notice any issues when resizing here, either.

[scottmcm]

Win 7, 2x NVIDIA GeForce GTX 550 Ti, 2560x1080

Default size or nearly-full: 16.0 or 17.0ms/frame (never saw .5). Consistently smooth.

Resizing at any height: ok, though some judder. Seems to gets a bit further from the top right widening compared to narrowing.

Note that idle is way better than master for me; on master I get consistent white at the right.

[UtherII]

Windows 7 - Intel HD Graphics 4600 -1680*1050

• Does the spinner spin smoothly? -> Yes
• Does the frame interval hold steady at 16-17ms? -> Yes
• Does it still perform well if the window is nearly fullscreen? -> Yes
• Grab the left side of the window and resize. Does the window frame track the mouse? -> Yes
• Is it smooth or is there jank? -> This is smooth while the windows size is changing. But when the mouse stop moving, the content is frozen until the mouse button is released.
• Does the diagonal line stay glued to the upper right corner or does it judder? -> There is indeed a small shift (a few pixel) during resize, especially if I resize fast.

[unchartedxx]

Windows 10, Intel HD 520, Surface PRO 4, external display 1440p with normal DPI and no scaling

I only checked on the current master branch.

• The spinner seems smooth
• The frame is at 15.6ms.
• The same at fullscreen
• Left border resize is glued to cursor
• The diagonal line is perfectly from corner to corner
• I just notice that if I move or resize the window, the spinner does not spin and the frame time reports strange values (from 0 to 100ms ) but I suppose it's normal.

[abonander]

Windows 10, GTX 1080 Ti, 1440p@60hz, no DPI scaling.

No visual issues on idle or flip_hwnd, completely smooth, no jank or desync of the animations that I can tell.

However the frame-time for flip_hwnd during idle is only 2ms which is way overkill for 60hz and has correspondingly high cpu usage at 10% on an 8C/16T CPU. When holding the resize handle but not moving, it hovers at the expected 16~17ms and actively resizing drops the frame-time to 4ms.

On idle the frame-time hovers at 16ms (2% CPU) during idle but drops to 4ms during active resize which causes a CPU usage spike, but it takes me actively jiggling the resize handle to get anywhere near flip_hwnd's idle CPU usage.

Interestingly enough, minimizing the window actually causes the highest CPU usage of all on the idle branch, hovering at 8% CPU (obviously I can't observe the frame time); flip_hwnd doesn't spike, it just maintains its 10% CPU.

Switching back to master, it performs comparably to idle as far as I can tell, even running hot when the window is minimized.

[hexjelly]

Windows 10, GTX 1050, 1080@60hz, no DPI scaling

Everything seems smooth and comparable on both branches.

On master the window content will freeze until mouse button is released (but will still change/render while actively resizing). It will also freeze the content while just moving the window (by holding mouse button on title bar).

On flip_hwnd it does not freeze in either instances, except for a slight ~1s initial freeze when holding mouse button down on the title bar. This does not happen when holding button down while resizing.

[iliekturtles]

Win 10, Intel(R) Xeon(R) CPU E3-1271 v3 @3.6GHz, NVIDIA Quadro K620 (1x display port -> display port hub -> 3 DVI monitors @ 1920x1080)

• Does the spinner spin smoothly? Yes
• Does the frame interval hold steady at 16-17ms? Yes
• Does it still perform well if the window is nearly fullscreen? Yes
• Does the window frame track the mouse? Yes, however the spinner only updates when dragging and the frame interval shows the time since the last drag event
• Is it smooth or is there jank? Smooth
• Does the diagonal line stay glued to the upper right corner or does it judder? Glued, I don't detect any judder
• Is there flashing or any artifact when entering and exiting sizing? No flashing, the only thing I notice is a change in the frame interval to 1 digit instead of 2, unlike master the spinner and frame interval stay running smooth
• Is performance roughly comparable? Better since spinner and frame interval update continuously

As @DanielKeep mentioned the border seems to be offset by a couple pixels. This is most noticeable when the application is maximized. It can also be detected when the window is not maximized. Note that the point becomes the drag icon a few pixels outside the border instead of when directly on the edge.

[Connicpu]

• Windows 10 1803 (build 17133.73)
• NVidia GTX 1080 Reference card, driver 391.35 (latest)
• 200% DPI Scaling

idle (#18) branch

• Smooth spinner, steady frame (16-17ms)
• No performance drop at near fullscreen (4k monitor)
• No jank, diagonal line stays glued.

flip_hwnd branch

• Smooth spinner, steady frame (16-17ms)
• No performance drop at near fullscreen (4k monitor)
• No jank, diagonal line stays glued.

Basically 0 issues for me, window seems to run perfectly with no visual problems

[Tembocs]

Windows 10 1803 (insider build 17133.73), GT 620, 1920 x 1080 @60Hz.

• Spinner spin smoothly.
• Frame interval hold steady around 15.6 ms.
• It does perform well if the window is nearly full screen

On resizing

• The window frame track the mouse with a bit of lag though it is smooth.
• The diagonal line stay glued to the upper right corner.

[doxxx]

• Dell Precision 5510 laptop
• Windows 10 build 16299
• Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz, 2701 Mhz, 4 Core(s), 8 Logical Processor(s)
• Intel(R) HD Graphics 530 / NVIDIA Quadro M1000M (not sure which one was in effect at the time)
• 1920 x 1080 x 59 hertz

master:

• Smooth spinner
• Steady framerate 16-17ms
• Window border tracks the mouse smoothly when resizing, only lags very slightly when moving the mouse very fast -- seems to be slightly more lag on the laptop screen vs the attached monitor (via Thunderbolt dock)
• Diagonal line stayed glued to the corners when resizing from any border
• No change in framerate or spinner when maximized
• When dragging the window border, if I stop moving the mouse while keeping the button pressed, the spinner stops. When I start moving the mouse again, the spinner starts again.
• I also noticed that the actual draggable area of the border is outside the window frame.
• I also noticed the maximized size is not quite filling the screen.
• CPU usage for the perftest process is ~7%

idle branch:

• same as master, but the spinner continues spinning when I stop moving the mouse while dragging
• less lag than master while dragging at high speeds

flip_hwnd branch:

• same as master, but the spinner continues spinning when I stop moving the mouse while dragging

[valpackett]

• Windows 10 1709
• AMD Ryzen 7 1700 @ 3.9GHz, DDR4-3333 CL16-15-15-36
• AMD Radeon RX 480, driver 23.20.15033.5003 (Crimson 18.3.4) (Mar 22, 2018)
• 3840x2160x60Hz, 150% DPI scaling

On both master and flip_hwnd, everything is perfect (17ms frametime, glued line, smooth spin, smooth resize, even when window is big).

CPU usage is ~2% when idle, ~2.3% when constantly resizing back and forth horizontally, ~2.7% diagonally.

[wezm]

Device Eve V 2 in 1
Processor Intel(R) Core(TM) i7-7Y75 CPU @ 1.30GHz, 1608 Mhz, 2 Core(s), 4 Logical Processor(s)
OS Name Microsoft Windows 10 Home
Version 10.0.16299 Build 16299
Name Intel(R) HD Graphics 615
Resolution 2880 x 1920 x 59 hertz
Scaling 200%

Does the spinner spin smoothly?

Yes

Does the frame interval hold steady at 16-17ms?

Yes

Does it still perform well if the window is nearly fullscreen?

Yes

Grab the left side of the window and resize. Does the window frame track the mouse?

Yes

Is it smooth or is there jank?

Smooth

Does the diagonal line stay glued to the upper right corner or does it judder?

Stays glued to right corner.

flip_hwnd

Same results.

[matrizzo]

Windows 10, Nvidia GTX 770, 1920x1080@120Hz, 100% DPI scaling.

Does the spinner spin smoothly?

Yes.

Does the frame interval hold steady at 16-17ms?

It's either steady at 8ms or jumps between 8ms and 15-16ms.

Does it still perform well if the window is nearly fullscreen?

Yes.

Grab the left side of the window and resize. Does the window frame track the mouse?

Yes.

Is it smooth or is there jank?

Smooth.

Does the diagonal line stay glued to the upper right corner or does it judder?

It stays glued to the upper right corner.

Same results with the flip_hwnd branch, no flashing or artifacts. Performance (as in frame time) is the same.

[chippers]

Windows 10 Pro x64, Nvidia GTX 1080ti, 2560x1440@144hz, 100% DPI scale

Does the spinner spin smoothly?

Yes

Does the frame interval hold steady at 16-17ms?

~6.9-7.0ms which is right on for 144hz.

Does it still perform well if the window is nearly fullscreen?

Yes

Grab the left side of the window and resize. Does the window frame track the mouse?

Yes

Is it smooth or is there jank?

Smooth

Does the diagonal line stay glued to the upper right corner or does it judder?

Stays glued to right corner.

---

Windows 10 Pro x64, Nvidia GTX 1080ti, 3140x2160@60hz, 125% DPI scale

Does the spinner spin smoothly?

Yes

Does the frame interval hold steady at 16-17ms?

Yes

Does it still perform well if the window is nearly fullscreen?

Yes

Grab the left side of the window and resize. Does the window frame track the mouse?

Yes

Is it smooth or is there jank?

The left side is smooth. The right side of the window janks and moves around by a pixel or 2 while the left is moving. It's all smooth on the flip_hwnd branch.

Does the diagonal line stay glued to the upper right corner or does it judder?

Stays glued to right corner.

Also, the text looks a bit blurry on the 4k due to the 125% DPI. Not as smooth as text in a browser window on the same monitor.

[raphlinus]

Thanks all! These results are encouraging enough I feel like I can move forward with the flip+hwnd hybrid approach. One of the main remaining problems, aside from cleanup, is making it load optionally, to preserve Windows 7 compatibility. I'm still interested in whether there's a better approach, but if there is one it's neither easy to figure out or already known to Windows developers.

The extra window border when maximized is because I had WS_EX_OVERLAPPEDWINDOW, removed in #20. Thanks for pointing it out.

The blurry text at 125% is because it's using system rather than per-monitor dpi. I have an experiment where I got per-monitor dpi working, but haven't merged it into xi-win yet. It's fairly tricky, generally they want you to put the dpi capabilities in the manifest, but I don't have one yet. Also, I'm not sure it's straightforward to include a manifest in example binaries, so for this it might be better to do it in code.

[trha]

Windows 10 17133.73, Intel HD Graphics 520 & Nvidia GTX 960M, UHD resolution (3840x2160@60Hz), 175% scaling.

Does the spinner spin smoothly? Does the frame interval hold steady at 16-17ms? Does it still perform well if the window is nearly fullscreen?

On the master branch: Frame rate is 60 FPS when the window is not too big, but drops to 30 FPS when the window is nearly full screen.

On the flip_hwnd branch: Frame rate is stable at 60 FPS, regardless of window size.

Grab the left side of the window and resize. Does the window frame track the mouse? Is it smooth or is there jank? Does the diagonal line stay glued to the upper right corner or does it judder?

Everything is super smooth when resizing on both branches. The window only flickers when being maximized from a small size.

[raphlinus]

Eager testers might want to check out #21, which is a cleaned up version. It should work on Windows 7 (using the hwnd approach only). Overall I expect it to perform pretty similarly, maybe a bit better (I removed some stray present operations that weren't needed when sizing).

[dd10-e]

Hi! Since this PR, the window resizing is really super smooth but there is a "bug" when i maximized with the windows button. We have to type on the keyboard for a return to normal colors. I'm pretty sure it was not like that in before !

With last commits since today on master.

My configuration :

• Microsoft Windows 10 Famille, 10.0.16299 Numéro 16299
• Intel(R) Core(TM)2 Quad CPU Q9300 @ 2.50GHz, 2500 MHz, 4 core, 4
• PL2283H (1920x1080@60Hz), 2047MB NVIDIA GeForce GT 730 (ASUStek Computer Inc)

Does the frame interval hold steady at 16-17ms?

Yes, 16-17ms, and when i resize it seems to get down to 9ms (very fast!).

Does it still perform well if the window is nearly fullscreen?

Same think as before.

Grab the left side of the window and resize. Does the window frame track the mouse?

Yes.

Is it smooth or is there jank?

So smooth !

Does the diagonal line stay glued to the upper right corner or does it judder?

It stay glued.

Otherwise, it's an amazing work! I'm super happy to see this project growing up.