Use Binary Archives as a pipeline cache on the metal backend #3719

expenses · 2021-04-07T15:57:18Z

See #3716. This PR builds on-top of gfx-rs/metal-rs#207 to add an implementation of get_pipeline_cache_data for the metal backend. This PR is not ready for reviews.

Todo:

Get rid of the Mutex around the pipeline cache and replace is_empty with an AtomicBool, as unsafe impl Sync is probably fine for the binary archive.
Check the availability of binary archives on the target with device.supports_binary_archive().
Potentially use a second cache to store SPIR-V -> MSL transformations.
Come up with a test suite to test pipeline caches. Something with hundreds/thousands of pipelines (maybe using shaders from shadertoy?) would be nice.

PR checklist:

make succeeds (on *nix)
make reftests succeeds
tested examples with the following backends: metal

expenses · 2021-04-09T19:35:46Z

I've added most of the features that I want for this now; what's left is a bit of a clean up, optimization as far as serialization is concerned and to merge gfx-rs/naga#686 and https://github.com/grovesNL/spirv_cross/compare/master...expenses:serialize-feature?expand=1.

expenses · 2021-04-11T09:48:41Z

Paused for a little bit because I'd really like the SPV->MSL cache to be done for naga, but testing this requires naga compilation of the shadertoy shaders to work a little better. gfx-rs/naga#693 fixes most of the issues.

kvark · 2021-04-12T13:24:11Z

This is also going to help gfx-portability. In particular, as we are finishing up the Naga fixes needed to get Dota2 running, we'll be benchmarking again, and having the working shader cache will boost it significantly.

expenses · 2021-04-13T21:36:26Z

src/backend/metal/src/device.rs

+        Ok(bincode::serialize(&n::SerializablePipelineCache {
+            binary_archive: &binary_archive()?,
+            spv_to_msl: n::serialize_spv_to_msl_cache(&cache.spv_to_msl),
+        })
+        .unwrap())


This could be made faster by not using bincode and making it zero-copy.

expenses · 2021-04-13T21:36:52Z

src/backend/metal/src/device.rs

+            let pipeline_cache: n::SerializablePipelineCache = bincode::deserialize(data).unwrap();
+
+            Ok(n::PipelineCache {
+                binary_archive: create_binary_archive(&pipeline_cache.binary_archive)?,
+                spv_to_msl: n::load_spv_to_msl_cache(pipeline_cache.spv_to_msl),
+            })


This could be made faster by not using bincode and making it zero-copy.

expenses · 2021-04-13T21:38:57Z

I think this is about ready for review now. gfx-rs/naga#716 needs to be merged and a gfx-22 tag of naga needs to be made before merging.

expenses · 2021-04-13T21:42:33Z

We're still parsing and validating the SPIR-V code with naga unnecessarily, as we should be able to skip over that if the SPIR-V is in the cache. Parsing and validating only seems to take about 40ms for 8000 modules though, so I don't think this is a huge problem.

expenses · 2021-04-13T21:48:14Z

Parsing and validating only seems to take about 40ms for 8000 modules though, so I don't think this is a huge problem.

Writing out the MSL shaders with a fresh pipeline cache also only takes 50ms for 8000 modules, so it appears that naga is fast enough that there's not too much benefit to having a SPV->MSL cache in the first place.

expenses · 2021-04-13T21:53:22Z

Writing out the MSL shaders with a fresh pipeline cache also only takes 50ms for 8000 modules, so it appears that naga is fast enough that there's not too much benefit to having a SPV->MSL cache in the first place.

Here are some timings for 8286 pipelines (less than the number in #3716 (comment) because some fail to verify):

cleared spv->msl cache: 8.50 5.87 5.44 5.46 6.42
hot spv->msl cache: 5.48 4.89 4.78 4.78 4.64

kvark

This is some top quality work, thank you for kicking this forward!

examples/quad/main.rs

kvark · 2021-04-14T03:56:16Z

src/backend/metal/Cargo.toml

@@ -15,7 +15,7 @@ edition = "2018"
 [features]
 default = []
 signpost = []
-cross = ["spirv_cross", "auxil", "naga/spv-out"]
+cross = ["spirv_cross", "auxil"]


why is naga/spv-out removed here?

Because we're now enabling it by default for create_shader_module_from_naga.

I've changed things around now and we only use naga/spv-out if we're using the pipeline-cache feature or the cross feature.

so for pipeline caching, we basically have to always go through SPIR-V? It's not the worst thing, I imagine wgpu will just not have this feature enabled for the near future.

Not necessarily, I could probably look into reworking things so we cache the hash of the WSL as well.

src/backend/metal/Cargo.toml

src/backend/metal/src/device.rs

src/backend/metal/src/native.rs

expenses · 2021-04-14T16:41:57Z

I timed it again and having a binary archive pipeline cache definitely makes a difference when the system cache has been cleaned.

wiped system cache, with binary archive: 293.24 288.50
wiped system cache, no binary archive: 892.77 889.07

kvark

This looks great!
I noted a few ideas here, but overall happy with this landing as soon as naga's gfx-22 train is up.

src/backend/metal/src/device.rs

src/backend/metal/src/native.rs

kvark · 2021-04-19T16:11:13Z

@expenses Naga's "gfx-22" is up, please update the PR to use it (and make sure every commit is testable).

kvark · 2021-04-19T17:16:40Z

@expenses can we make it so every commit is testable? If not sure, let's squash everything.

expenses · 2021-04-19T17:17:38Z

Okay, I'll do a squash then I think.

expenses · 2021-04-19T17:23:27Z

I'm trying to find a good way to do with while having the same commit message that doing a squash merge on github does. Maybe just doing it through the site would be better?

1339: Update naga to gfx-22, add interpolation sampling validation r=kvark a=kvark **Connections** Includes gfx-rs/gfx#3729, gfx-rs/gfx#3719, gfx-rs/naga#689, gfx-rs/naga#700, and other fixes **Description** Update gfx and naga to latest (gfx-22 tag). The hope is for this to be the last tag release before 0.8 **Testing** Tested on wgpu-rs examples Co-authored-by: Dzmitry Malyshau <[email protected]>

expenses added 9 commits April 6, 2021 14:37

First draft of a pipeline cache for metal using binary archives!

a35f8d0

Load pipeline caches from disk!

4c2fb06

Clean up quad example a little bit

3cf9016

Add warning to merge_pipeline_caches

c36ae38

Handle empty data inputs to create_pipeline_cache

0eb9340

Tested that pipeline caching does indeed work!

c5bcd27

Do the same for compute pipelines

39b9bdc

Make the binary archive optional based on features

a1063dd

Don't use a Mutex for the BinaryArchive.

f729ce6

expenses mentioned this pull request Apr 9, 2021

Pipeline cache serialization/deserialization investigation #3716

Open

Add a spv->msl cache

fb47035

expenses mentioned this pull request Apr 9, 2021

Add Serialize/Deserialize derives for mtl::EntryPointError gfx-rs/naga#686

Merged

expenses added 2 commits April 9, 2021 20:13

Switch to git patches

fbc4387

Serialize/Deserialize both caches

f74a293

Merge remote-tracking branch 'origin/master' into metal-pipeline-cache

9294a13

expenses mentioned this pull request Apr 13, 2021

Impl Hash for msl::Options gfx-rs/naga#716

Merged

expenses added 2 commits April 13, 2021 23:11

use a spv->msl cache for naga instead.

6aabde7

Use a git patch for naga instead of a path

d0e7e40

expenses commented Apr 13, 2021

View reviewed changes

expenses marked this pull request as ready for review April 13, 2021 21:38

expenses changed the title ~~[Draft] Use Binary Archives as a pipeline cache on the metal backend~~ Use Binary Archives as a pipeline cache on the metal backend Apr 13, 2021

kvark reviewed Apr 14, 2021

View reviewed changes

expenses added 2 commits April 14, 2021 19:22

Apply some suggestions

6344e31

Make a lot of things dependent on a pipeline-cache feature

fd6ff4e

expenses requested a review from kvark April 14, 2021 17:25

expenses added 2 commits April 17, 2021 19:56

Update naga

034a78e

Merge remote-tracking branch 'origin/master' into metal-pipeline-cache

e0b76d9

expenses mentioned this pull request Apr 17, 2021

Derive Eq for msl::Options gfx-rs/naga#725

Merged

Use latest naga on the gl and vulkan backends

9e27f52

kvark approved these changes Apr 19, 2021

View reviewed changes

src/backend/metal/src/device.rs Outdated Show resolved Hide resolved

src/backend/metal/src/device.rs Outdated Show resolved Hide resolved

src/backend/metal/src/native.rs Outdated Show resolved Hide resolved

kvark mentioned this pull request Apr 19, 2021

[meta] Process Dota2's SPIRV to MSL gfx-rs/naga#409

Closed

expenses added 3 commits April 19, 2021 17:49

Merge remote-tracking branch 'origin/master' into metal-pipeline-cache

cdf771c

Apply suggestions wrt u8s

fab2b4a

Move pipeline cache stuff to its own module

9dbf6b7

Switch to naga gfx-22

ce89c9c

kvark merged commit 6999171 into gfx-rs:master Apr 19, 2021

kvark mentioned this pull request Apr 19, 2021

Update naga to gfx-22, add interpolation sampling validation gfx-rs/wgpu#1339

Merged

expenses mentioned this pull request Apr 21, 2021

Update all dependencies repi/shadertoy-browser#62

Merged

kvark mentioned this pull request May 11, 2021

New Metal pipeline cache doesn't work properly on Dota2 + portability #3755

Open

hgallagher1993 mentioned this pull request May 13, 2021

Pipeline caches on DX12 #2877

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use Binary Archives as a pipeline cache on the metal backend #3719

Use Binary Archives as a pipeline cache on the metal backend #3719

expenses commented Apr 7, 2021 •

edited

Loading

expenses commented Apr 9, 2021

expenses commented Apr 11, 2021

kvark commented Apr 12, 2021

expenses Apr 13, 2021

expenses Apr 13, 2021

expenses commented Apr 13, 2021

expenses commented Apr 13, 2021

expenses commented Apr 13, 2021

expenses commented Apr 13, 2021 •

edited

Loading

kvark left a comment

kvark Apr 14, 2021

expenses Apr 14, 2021

expenses Apr 14, 2021

kvark Apr 14, 2021

expenses Apr 14, 2021

expenses commented Apr 14, 2021

kvark left a comment

kvark commented Apr 19, 2021

kvark commented Apr 19, 2021

expenses commented Apr 19, 2021

expenses commented Apr 19, 2021

Use Binary Archives as a pipeline cache on the metal backend #3719

Use Binary Archives as a pipeline cache on the metal backend #3719

Conversation

expenses commented Apr 7, 2021 • edited Loading

expenses commented Apr 9, 2021

expenses commented Apr 11, 2021

kvark commented Apr 12, 2021

expenses Apr 13, 2021

Choose a reason for hiding this comment

expenses Apr 13, 2021

Choose a reason for hiding this comment

expenses commented Apr 13, 2021

expenses commented Apr 13, 2021

expenses commented Apr 13, 2021

expenses commented Apr 13, 2021 • edited Loading

kvark left a comment

Choose a reason for hiding this comment

kvark Apr 14, 2021

Choose a reason for hiding this comment

expenses Apr 14, 2021

Choose a reason for hiding this comment

expenses Apr 14, 2021

Choose a reason for hiding this comment

kvark Apr 14, 2021

Choose a reason for hiding this comment

expenses Apr 14, 2021

Choose a reason for hiding this comment

expenses commented Apr 14, 2021

kvark left a comment

Choose a reason for hiding this comment

kvark commented Apr 19, 2021

kvark commented Apr 19, 2021

expenses commented Apr 19, 2021

expenses commented Apr 19, 2021

expenses commented Apr 7, 2021 •

edited

Loading

expenses commented Apr 13, 2021 •

edited

Loading