Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor canvas context to allow presenting as image #586

Merged
merged 27 commits into from
Sep 21, 2024
Merged

Conversation

almarklein
Copy link
Member

@almarklein almarklein commented Sep 16, 2024

Background

This is a first step in the "update propagation" work. One of the goals of that work is to clearly separate rendering from events. And thus webgpu API from canvas API. A thought that played through my mind was to implement the wgpu.gui in a separate package. I don't think I want to go that far (for now), but I definitely want to avoid leaking any wgpu-specifics into the gui subpackage.

Problem

The CanvasContext is the thing that ties webgpu and the gui layer together. It enables webgpu to draw into a GUI window. This works ok as it is, but e.g. the offscreen canvas implements its own version of a CanvasContext, which is an abstraction leak.

Proposal

This PR refactors the CanvasContext as well as the WgpuCanvasInterface so that the canvas can indicate how it wants the canvas to draw into it. It can either let it draw to the screen, or provide the canvas with the resulting image.

This makes offscreen canvases less special, so they won't have to hack their own fake context. It also allows other canvases to use images where it makes sense, e.g. a Tk backend, or Qt on Wayland, etc.

Details

The WgpuCanvasInterface.get_surface_info() returns a dict. That dict has at least a "method" field, which is either "screen" or "image". When it is "screen", it also provides "window" (and "display" too on Linux), like it did before. The dict can also provide capabilities like "formats" to indicate supported texture formats.

The WgpuCanvasInterface.present_image() method must be implemented if a canvas wants to support the image method.

Benchmarks

FPS measurements (with max_fps=999). Shown are fps for screen / image. Using a light scene (cube) and a heavy one (sponza).

640x480 cube fullscreen cube 640x480 sponza fullscreen sponza
MacOS M1 120 / 130 110 / 33 28 / 22 23 / 14
Windows with Geforce RTX 2070 144 / 105 67 / 47 9 / 9 8 / 8
Linux with Intel UHD 730 60 / 190 60 / 38 17 / 9 4 / 3
Windows with Intel UHD 730 on Vulkan 60 / 150 56 / 37 23 / 14 10 / 7
Windows with Intel UHD 730 on D3D12 60 / 120 56 / 35 na na

Observations:

  • There's certainly a performance penalty for rendering via an image.
  • The penalty scales with the size of the window, as expected.
  • When drawing to screen, we can observe an fps limit that's not present when drawing via an image.
  • My Windows laptop's GPU is rubbish?

Remarks:

  • This was tested with Qt. I would expect that Qt renders the image to screen really fast, but other solutions might do it faster.

Tasks

  • Basics working with Qt
  • Do performance benchmarks to have a better idea of the penalty for using the image method.
  • Remove flicker on Windows (introduced in (earlier version of) this pr).
  • Fix panic when process ends.
  • Can we get rid of CanvasContext.present(), since its not official webgpu api? later
  • Adjust other backends.
  • Use this as a workaround to fix Qt support on Wayland.
  • Clean up / check todos.
  • Update status in Linux Wayland support #92

Changelog

  • The WgpuOffscreenCanvasBase class is removed. Offscreen canvases can be implemented by returning the appropriate dict in canvas.get_surface_info() and implementing canvas.present_image().
  • The canvas.get_surface_info() method is renamed canvas.get_present_info() and must the returned dict has new specifications.
  • The canvas.present_image() method can be implemented to support presenting via an image.
  • The Qt and wx backends now also work on Wayland (though with a performance penalty).

@almarklein almarklein changed the title Refactor canvas context to allow presening as image Refactor canvas context to allow presenting as image Sep 16, 2024
@almarklein
Copy link
Member Author

I have more plans with the GUI subpackage, but I'll apply/propose these in separate pr's.

@almarklein almarklein marked this pull request as ready for review September 18, 2024 09:22
@almarklein
Copy link
Member Author

@Korijn this is ready for review.

Copy link
Collaborator

@Korijn Korijn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvements!

@almarklein almarklein merged commit 379599c into main Sep 21, 2024
23 checks passed
@almarklein almarklein deleted the canvascontext branch September 21, 2024 20:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants