[meta] Process Dota2's SPIRV to MSL

[kvark]

We'd want to run Dota2 with gfx-portability on Metal in a way that doesn't involve SPIRV-Cross at all. Will file separate issues for the SPV-in and MSL-out until we fully cover this path.

Test set - dota2-shaders.zip
My bash script to bulk-convert them:

NAGA_DIR=<insert naga path here>
NAGA_BIN=${NAGA_DIR}/target/debug/convert
(cd ${NAGA_DIR} && cargo build --features spv-in,msl-out,serialize)
for file in *.spv ; do \
	echo "Translating" ${file}; \
	$NAGA_BIN ${file} >${file}.txt
	$NAGA_BIN ${file} ${file}.metal
	#tail -n +5 ${file} | xcrun -sdk macosx metal -x metal - -o /dev/null; \
done

I launch it as "sh ../naga.sh" from the folder with SPVs.

[kvark]

Here is my small script:

NAGA=../../naga/target/debug/examples/convert
(cd ../../naga && cargo build --example convert --features spv-in,msl-out)
for file in spirv/*.spv ; do \
	echo "Translating" ${file}; \
	$NAGA ${file} ${file}.metal
	#xcrun -sdk macosx metal -mmacosx-version-min=10.11 ${file}.metal - -o /dev/null; \
done

[kvark]

Here is the first log, up to shader-229, which hangs because of another problem that I'm about to address.

[2021-02-09T15:33:08Z ERROR naga::back::msl] Missing binding for Some("")
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Validation', examples/convert.rs:193:65
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedBinaryOp(LogicalOr)', examples/convert.rs:193:65
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedCapability(ClipDistance)', examples/convert.rs:95:64
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedInstruction(Type, ConstantFalse)', examples/convert.rs:95:64
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedCapability(ClipDistance)', examples/convert.rs:95:64
Translating spirv/111.spv
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedInstruction(Type, ConstantFalse)', examples/convert.rs:95:64
Translating spirv/112.spv
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedCapability(ClipDistance)', examples/convert.rs:95:64
Translating spirv/113.spv
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedInstruction(Type, ConstantFalse)', examples/convert.rs:95:64
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedBinaryOp(LogicalOr)', examples/convert.rs:193:65
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedInstruction(Type, ConstantFalse)', examples/convert.rs:95:64
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedBinaryOp(LogicalOr)', examples/convert.rs:193:65
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedCapability(ClipDistance)', examples/convert.rs:95:64
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedInstruction(Type, ConstantFalse)', examples/convert.rs:95:64
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedCapability(ClipDistance)', examples/convert.rs:95:64
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedInstruction(Type, ConstantFalse)', examples/convert.rs:95:64
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedBinaryOp(LogicalOr)', examples/convert.rs:193:65
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedBinaryOp(LogicalOr)', examples/convert.rs:193:65
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedCapability(ClipDistance)', examples/convert.rs:95:64
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedInstruction(Type, ConstantFalse)', examples/convert.rs:95:64
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedCapability(ClipDistance)', examples/convert.rs:95:64
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedInstruction(Type, ConstantFalse)', examples/convert.rs:95:64
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedCapability(ClipDistance)', examples/convert.rs:95:64
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedInstruction(Type, ConstantFalse)', examples/convert.rs:95:64
thread 'main' panicked at 'Global variable Handle(12) '' is invalid: InvalidBuiltInType(FrontFacing)', examples/convert.rs:54:31
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedInstruction(Function, VectorExtractDynamic)', examples/convert.rs:95:64
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedInstruction(Type, ConstantFalse)', examples/convert.rs:95:64
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UnsupportedInstruction(Type, ConstantFalse)', examples/convert.rs:95:64
thread 'main' panicked at 'Entry point main at Vertex is invalid: Function(Resolve(IncompatibleOperands { op: "x", left: "Vector { size: Quad, kind: Float, width: 4 }", right: "Vector { size: Tri, kind: Float, width: 4 }" }))', examples/convert.rs:54:31

I think it looks fairly small so far: we are very close to be able to get this going!

[kvark]

With #478, we are able to actually process all the shaders!
Remaining issues are in this order:

1. Missing bindings: it's not a bug, strictly speaking, but it would be nice to have a mode where the bindings are automatically assigned, so that we can test the shaders outside of gfx/wgpu.
2. Uniform control flow validation error. It's either a bug in the validation, or we need different profiles for validating WebGPU versus Vulkan, to make this softer.
3. Use of the gl_PerVertex interface block - [meta] glslang-generated SPIR-V issues #475

[kvark]

(1) is addressed in #500

There is also (4) - #504, which is addressed in #505

[kvark]

I think we are getting very close now with #587!
We are successfully compiling 366 / 423 shaders. I haven't checked if they are correct Metal sources, but that's looking promising.
The remaining issues though - #586 and gfx-rs/wgpu#4317 appear more deep and complicated.

[kvark]

I'm trying to make progress with the actual MSL now. Not just validating that IR is correct, but actually running gfx-portability with that generated MSL, and I'm seeing more issues:

• a bunch is fixed in Various MSL fixes: address spaces, select, brackets, constants, and others #615
• SPIR-V frontend emits some expressions multiple times #613
• MSL struct initialization fails on array types #614

[kvark]

New issues - #620, #623, while #619 closes one of the previous issues.

[kvark]

Good news - with #641, all the 403 shaders (up to the main menu) are successfully transpiled, and are valid Metal shaders.
Bad news - everything is upside down 🙃 - gfx-rs/gfx#3707

[kvark]

Looking at the differences with generated code now.

Empty switch

This one is cosmetic only.
Naga:

switch(as_type<int>(const_0u)) {
                    default: {...
                    }
}

cross:

do
            {...
} while (false);

Default and a branch

This appears to be a change in logic.
Naga:

                            default: {
                            }
                        }
                        phi_18521_ = phi_11219_;
                        if (phi_14099_) {
                        }
                        phi_18521_ = const_1f;

Cross:

                    default:
                    {
                        _11219 = _2;
                        _14099 = false;
                        break;
                    }
                }
                if (_14099)
                {
                    _18521 = _11219;
                    break;
                }
                _18521 = 1.0;

Edit: both can be found in
112.spv.zip

[kvark]

Looks like #683 is the last issue here.

[kvark]

Getting ready to dance with gfx-rs/gfx#3732
Benchmarking will start once #683 is fixed, and gfx-rs/gfx#3719 is landed.