3DS: Initial accelerated renderer

[ccawley2011]

This is a rebased version of @asiekierka's original branch from #6251 (comment), with some additional fixes to get testgeometry to partially work. testgeometry now displays, however the viewport is incorrect and it hangs when exiting the application. The rest of the test programs don't produce useful results.

I'm not that familiar with 3DS GPU programming (or GPU programming in general), so any help would be appreciated.

[asiekierka]

Thank you for your efforts! After debugging, I found out that this was down to an incorrect implementation of RenderSetViewport; attached a patch below which makes testgeometry render correctly:

diff --git a/src/render/n3ds/SDL_render_n3ds.c b/src/render/n3ds/SDL_render_n3ds.c
index bb26480ea..a38586448 100644
--- a/src/render/n3ds/SDL_render_n3ds.c
+++ b/src/render/n3ds/SDL_render_n3ds.c
@@ -666,8 +666,15 @@ N3DS_RunCommandQueue(SDL_Renderer * renderer, SDL_RenderCommand *cmd, void *vert
         switch (cmd->command) {
             case SDL_RENDERCMD_SETVIEWPORT: {
                 SDL_Rect *viewport = &cmd->data.viewport.rect;
-                C3D_SetViewport(viewport->x, viewport->y, viewport->w, viewport->h);
-                C3D_SetScissor(GPU_SCISSOR_NORMAL, viewport->x, viewport->y, viewport->x + viewport->w, viewport->y + viewport->h);
+                if (data->boundTarget) {
+                    C3D_SetViewport(viewport->x, viewport->y, viewport->w, viewport->h);
+                } else {
+                    // Handle the tilted render target of the 3DS display.
+                    C3D_SetViewport(
+                               data->renderTarget->frameBuf.width - viewport->h - viewport->y,
+                               data->renderTarget->frameBuf.height - viewport->w - viewport->x,
+                               viewport->h, viewport->w);
+                }
                 break;
             }

@@ -687,7 +694,7 @@ N3DS_RunCommandQueue(SDL_Renderer * renderer, SDL_RenderCommand *cmd, void *vert

             case SDL_RENDERCMD_SETCLIPRECT: {
                 const SDL_Rect *rect = &cmd->data.cliprect.rect;
-                if(cmd->data.cliprect.enabled){
+                if (cmd->data.cliprect.enabled) {
                     C3D_SetScissor(GPU_SCISSOR_NORMAL, rect->x, rect->y, rect->x + rect->w, rect->y + rect->h);
                 } else {
                     C3D_SetScissor(GPU_SCISSOR_DISABLE, 0, 0, 0, 0);

[asiekierka]

rect-patch.txt

And here's a further patch that makes rectangle drawing work properly. This gets testdraw2 closer to functional, at least, but not all lines render properly.

[ccawley2011]

Cool, thanks. I've added both patches to the branch.

A couple of additional notes:

• The viewport is now much better, however it's currently upside down compared to OpenGL. testgeometry for example should have the blue corner at the top.
• I think the remaining issue with rectangle drawing has something to do with batching. Using a modified version of testdrawchessboard shows that it works correctly with a single call to SDL_RenderFillRects, but not with multiple calls to SDL_RenderFillRect.

[asiekierka]

3ds_patch2.txt

This should fix the viewport being upside down. The issue with testdrawchessboard was actually related to it using a software renderer and then drawing its results on-screen with the accelerated renderer, which led to (a) incorrect U/V coordinates, (b) 24-bit textures not being properly swizzled. Both should now be improved.

[ccawley2011]

The issue with testdrawchessboard was actually related to it using a software renderer and then drawing its results on-screen with the accelerated renderer, which led to (a) incorrect U/V coordinates, (b) 24-bit textures not being properly swizzled. Both should now be improved.

I hadn't actually tried it without the modifications, so that looks like a separate issue. I've pushed the relevant changes to get testdrawchessboard to use the accelerated APIs which should demonstrate the issue I encountered.

[slouken]

I've pushed the relevant changes to get testdrawchessboard to use the accelerated APIs which should demonstrate the issue I encountered.

Please remove those changes before this PR is ready for review. testdrawchessboard is the only window surface test we have, and rely on it to catch regressions.

[asiekierka]

3ds_patch3.txt

Indeed, batching was broken. As was updating textures (I haven't figured out all the edge cases for hardware-accelerated texture swizzling, so I'm disabling it - for an initial implementation, this is probably good enough).

This seems to make all the simpler tests I've thrown at it work, now - testrendertarget remains broken.

EDIT: 3ds_patch4.txt - fixes a few minor issues, but not render targets.

Apologies for the pull request spam; I initially wrote this code in a day to get the general approach right, and I didn't realize it was quite this wonky...

[ccawley2011]

3ds_patch3.txt

EDIT: 3ds_patch4.txt - fixes a few minor issues, but not render targets.

Thanks. I've added both of these to the branch, along with a couple of extra fixes.

Please remove those changes before this PR is ready for review. testdrawchessboard is the only window surface test we have, and rely on it to catch regressions.

I've split this into a separate PR: #9664