Unify shared infra #2354

alliepiper · 2024-09-03T18:31:26Z

Description

This PR centralizes shared infrastructure to the CCCL level, reducing redundancy across our projects and facilitating the introduction of new subprojects.

miscco

I like it a lot

cmake/CCCLBuildCompilerTargets.cmake

cudax/cmake/cudaxBuildCompilerTargets.cmake

github-actions · 2024-09-17T05:50:36Z

🟨 CI finished in 7h 50m: Pass: 84%/33 | Total: 16h 37m | Avg: 30m 13s | Max: 1h 42m | Hits: 98%/5962

🟨 cub: Pass: 81%/11 | Total: 13h 07m | Avg: 1h 11m | Max: 1h 42m | Hits: 99%/727

🚨 jobs: TestGPU 🚨
  🟩 Build              Pass: 100%/3   | Total:  1h 45m | Avg: 35m 15s | Max: 47m 12s | Hits:  99%/727   
  🟩 DeviceLaunch       Pass: 100%/2   | Total:  2h 46m | Avg:  1h 23m | Max:  1h 25m
  🟩 GraphCapture       Pass: 100%/2   | Total:  2h 40m | Avg:  1h 20m | Max:  1h 20m
  🟩 HostLaunch         Pass: 100%/2   | Total:  2h 44m | Avg:  1h 22m | Max:  1h 23m
  🔥 TestGPU            Pass:   0%/2   | Total:  3h 10m | Avg:  1h 35m | Max:  1h 42m
🟨 cxx
  🟨 Clang18            Pass:  80%/5   | Total:  6h 23m | Avg:  1h 16m | Max:  1h 27m
  🟨 GCC13              Pass:  80%/5   | Total:  6h 29m | Avg:  1h 17m | Max:  1h 42m
  🟩 MSVC14.39          Pass: 100%/1   | Total: 14m 19s | Avg: 14m 19s | Max: 14m 19s | Hits:  99%/727   
🟨 cxx_family
  🟨 Clang              Pass:  80%/5   | Total:  6h 23m | Avg:  1h 16m | Max:  1h 27m
  🟨 GCC                Pass:  80%/5   | Total:  6h 29m | Avg:  1h 17m | Max:  1h 42m
  🟩 MSVC               Pass: 100%/1   | Total: 14m 19s | Avg: 14m 19s | Max: 14m 19s | Hits:  99%/727   
🟨 cpu
  🟨 amd64              Pass:  81%/11  | Total: 13h 07m | Avg:  1h 11m | Max:  1h 42m | Hits:  99%/727   
🟨 ctk
  🟨 12.6               Pass:  81%/11  | Total: 13h 07m | Avg:  1h 11m | Max:  1h 42m | Hits:  99%/727   
🟨 cudacxx
  🟨 nvcc12.6           Pass:  81%/11  | Total: 13h 07m | Avg:  1h 11m | Max:  1h 42m | Hits:  99%/727   
🟨 cudacxx_family
  🟨 nvcc               Pass:  81%/11  | Total: 13h 07m | Avg:  1h 11m | Max:  1h 42m | Hits:  99%/727   
🟨 gpu
  🟨 v100               Pass:  81%/11  | Total: 13h 07m | Avg:  1h 11m | Max:  1h 42m | Hits:  99%/727

🟨 libcudacxx: Pass: 60%/5 | Total: 54m 30s | Avg: 10m 54s | Max: 21m 28s | Hits: 99%/2903

🚨 jobs: Test 🚨
  🟩 Build              Pass: 100%/3   | Total: 41m 54s | Avg: 13m 58s | Max: 21m 28s | Hits:  99%/2903  
  🔥 Test               Pass:   0%/2   | Total: 12m 36s | Avg:  6m 18s | Max:  6m 23s
🟨 cxx
  🟨 Clang18            Pass:  50%/2   | Total: 27m 51s | Avg: 13m 55s | Max: 21m 28s
  🟨 GCC13              Pass:  50%/2   | Total: 10m 29s | Avg:  5m 14s | Max:  6m 13s
  🟩 MSVC14.39          Pass: 100%/1   | Total: 16m 10s | Avg: 16m 10s | Max: 16m 10s | Hits:  99%/2903  
🟨 cxx_family
  🟨 Clang              Pass:  50%/2   | Total: 27m 51s | Avg: 13m 55s | Max: 21m 28s
  🟨 GCC                Pass:  50%/2   | Total: 10m 29s | Avg:  5m 14s | Max:  6m 13s
  🟩 MSVC               Pass: 100%/1   | Total: 16m 10s | Avg: 16m 10s | Max: 16m 10s | Hits:  99%/2903  
🟨 cpu
  🟨 amd64              Pass:  60%/5   | Total: 54m 30s | Avg: 10m 54s | Max: 21m 28s | Hits:  99%/2903  
🟨 ctk
  🟨 12.6               Pass:  60%/5   | Total: 54m 30s | Avg: 10m 54s | Max: 21m 28s | Hits:  99%/2903  
🟨 cudacxx
  🟨 nvcc12.6           Pass:  60%/5   | Total: 54m 30s | Avg: 10m 54s | Max: 21m 28s | Hits:  99%/2903  
🟨 cudacxx_family
  🟨 nvcc               Pass:  60%/5   | Total: 54m 30s | Avg: 10m 54s | Max: 21m 28s | Hits:  99%/2903  
🟨 gpu
  🟨 v100               Pass:  60%/5   | Total: 54m 30s | Avg: 10m 54s | Max: 21m 28s | Hits:  99%/2903

🟨 thrust: Pass: 87%/8 | Total: 1h 48m | Avg: 13m 31s | Max: 26m 56s | Hits: 99%/2230

🔍 cxx: MSVC14.39 🔍
  🟩 Clang18            Pass: 100%/3   | Total: 39m 22s | Avg: 13m 07s | Max: 26m 56s
  🟩 GCC13              Pass: 100%/3   | Total: 39m 34s | Avg: 13m 11s | Max: 26m 43s
  🔍 MSVC14.39          Pass:  50%/2   | Total: 29m 16s | Avg: 14m 38s | Max: 19m 41s | Hits:  99%/2230  
🔍 cxx_family: MSVC 🔍
  🟩 Clang              Pass: 100%/3   | Total: 39m 22s | Avg: 13m 07s | Max: 26m 56s
  🟩 GCC                Pass: 100%/3   | Total: 39m 34s | Avg: 13m 11s | Max: 26m 43s
  🔍 MSVC               Pass:  50%/2   | Total: 29m 16s | Avg: 14m 38s | Max: 19m 41s | Hits:  99%/2230  
🔍 jobs: TestCPU 🔍
  🟩 Build              Pass: 100%/3   | Total: 30m 14s | Avg: 10m 04s | Max: 19m 41s | Hits:  99%/2230  
  🔍 TestCPU            Pass:  66%/3   | Total: 24m 19s | Avg:  8m 06s | Max:  9m 35s
  🟩 TestGPU            Pass: 100%/2   | Total: 53m 39s | Avg: 26m 49s | Max: 26m 56s
🟨 cpu
  🟨 amd64              Pass:  87%/8   | Total:  1h 48m | Avg: 13m 31s | Max: 26m 56s | Hits:  99%/2230  
🟨 ctk
  🟨 12.6               Pass:  87%/8   | Total:  1h 48m | Avg: 13m 31s | Max: 26m 56s | Hits:  99%/2230  
🟨 cudacxx
  🟨 nvcc12.6           Pass:  87%/8   | Total:  1h 48m | Avg: 13m 31s | Max: 26m 56s | Hits:  99%/2230  
🟨 cudacxx_family
  🟨 nvcc               Pass:  87%/8   | Total:  1h 48m | Avg: 13m 31s | Max: 26m 56s | Hits:  99%/2230  
🟨 gpu
  🟨 v100               Pass:  87%/8   | Total:  1h 48m | Avg: 13m 31s | Max: 26m 56s | Hits:  99%/2230

🟩 cudax: Pass: 100%/5 | Total: 27m 52s | Avg: 5m 34s | Max: 11m 32s | Hits: 23%/102

🟩 cpu
  🟩 amd64              Pass: 100%/5   | Total: 27m 52s | Avg:  5m 34s | Max: 11m 32s | Hits:  23%/102   
🟩 ctk
  🟩 12.6               Pass: 100%/5   | Total: 27m 52s | Avg:  5m 34s | Max: 11m 32s | Hits:  23%/102   
🟩 cudacxx
  🟩 nvcc12.6           Pass: 100%/5   | Total: 27m 52s | Avg:  5m 34s | Max: 11m 32s | Hits:  23%/102   
🟩 cudacxx_family
  🟩 nvcc               Pass: 100%/5   | Total: 27m 52s | Avg:  5m 34s | Max: 11m 32s | Hits:  23%/102   
🟩 cxx
  🟩 Clang18            Pass: 100%/2   | Total:  8m 05s | Avg:  4m 02s | Max:  5m 58s
  🟩 GCC13              Pass: 100%/2   | Total:  8m 15s | Avg:  4m 07s | Max:  5m 42s
  🟩 MSVC14.39          Pass: 100%/1   | Total: 11m 32s | Avg: 11m 32s | Max: 11m 32s | Hits:  23%/102   
🟩 cxx_family
  🟩 Clang              Pass: 100%/2   | Total:  8m 05s | Avg:  4m 02s | Max:  5m 58s
  🟩 GCC                Pass: 100%/2   | Total:  8m 15s | Avg:  4m 07s | Max:  5m 42s
  🟩 MSVC               Pass: 100%/1   | Total: 11m 32s | Avg: 11m 32s | Max: 11m 32s | Hits:  23%/102   
🟩 gpu
  🟩 v100               Pass: 100%/5   | Total: 27m 52s | Avg:  5m 34s | Max: 11m 32s | Hits:  23%/102   
🟩 jobs
  🟩 Build              Pass: 100%/3   | Total: 16m 12s | Avg:  5m 24s | Max: 11m 32s | Hits:  23%/102   
  🟩 Test               Pass: 100%/2   | Total: 11m 40s | Avg:  5m 50s | Max:  5m 58s

🟩 cccl: Pass: 100%/4 | Total: 19m 20s | Avg: 4m 50s | Max: 5m 43s

🟩 cpu
  🟩 amd64              Pass: 100%/4   | Total: 19m 20s | Avg:  4m 50s | Max:  5m 43s
🟩 ctk
  🟩 11.1               Pass: 100%/2   | Total:  8m 29s | Avg:  4m 14s | Max:  4m 22s
  🟩 12.6               Pass: 100%/2   | Total: 10m 51s | Avg:  5m 25s | Max:  5m 43s
🟩 cudacxx
  🟩 nvcc11.1           Pass: 100%/2   | Total:  8m 29s | Avg:  4m 14s | Max:  4m 22s
  🟩 nvcc12.6           Pass: 100%/2   | Total: 10m 51s | Avg:  5m 25s | Max:  5m 43s
🟩 cudacxx_family
  🟩 nvcc               Pass: 100%/4   | Total: 19m 20s | Avg:  4m 50s | Max:  5m 43s
🟩 cxx
  🟩 Clang9             Pass: 100%/1   | Total:  4m 22s | Avg:  4m 22s | Max:  4m 22s
  🟩 Clang18            Pass: 100%/1   | Total:  5m 08s | Avg:  5m 08s | Max:  5m 08s
  🟩 GCC6               Pass: 100%/1   | Total:  4m 07s | Avg:  4m 07s | Max:  4m 07s
  🟩 GCC13              Pass: 100%/1   | Total:  5m 43s | Avg:  5m 43s | Max:  5m 43s
🟩 cxx_family
  🟩 Clang              Pass: 100%/2   | Total:  9m 30s | Avg:  4m 45s | Max:  5m 08s
  🟩 GCC                Pass: 100%/2   | Total:  9m 50s | Avg:  4m 55s | Max:  5m 43s
🟩 gpu
  🟩 v100               Pass: 100%/4   | Total: 19m 20s | Avg:  4m 50s | Max:  5m 43s
🟩 jobs
  🟩 Infra              Pass: 100%/4   | Total: 19m 20s | Avg:  4m 50s | Max:  5m 43s

🛠️ Override Matrix

[
  {
    "jobs": [
      "test"
    ],
    "project": [
      "libcudacxx",
      "thrust",
      "cub",
      "cudax"
    ],
    "ctk": "curr",
    "cxx": [
      "gcc",
      "clang",
      "msvc"
    ]
  },
  {
    "jobs": [
      "infra"
    ],
    "project": "cccl",
    "ctk": "11.1",
    "cxx": [
      "gcc6",
      "clang9"
    ]
  },
  {
    "jobs": [
      "infra"
    ],
    "project": "cccl",
    "ctk": "curr",
    "cxx": [
      "gcc",
      "clang"
    ]
  }
]

👃 Inspect Changes

Modifications in project?

	Project
+/-	CCCL Infrastructure
+/-	libcu++
+/-	CUB
+/-	Thrust
+/-	CUDA Experimental
	pycuda
	CUDA C Core Library

Modifications in project or dependencies?

	Project
+/-	CCCL Infrastructure
+/-	libcu++
+/-	CUB
+/-	Thrust
+/-	CUDA Experimental
+/-	pycuda
+/-	CUDA C Core Library

🏃‍ Runner counts (total jobs: 33)

#	Runner
18	`linux-amd64-gpu-v100-latest-1`
10	`linux-amd64-cpu16`
5	`windows-amd64-cpu16`

These tests check that consumers can add_subdirectory(cccl/cub) add_subdirectory(cccl/thrust) to add Thrust or CUB to their build. This hasn't been sensible or supported since we migrated to the monorepo. Now that we're moving shared CMake infra to the CCCL, these tests are finally actually failing.

… helper to reduce redundancy.

These tests are redundant with CCCL-level tests, and no longer function now that install rules are centralized at the top level.

[skip-vdc][skip-docs][skip-rapids]

alliepiper requested review from a team as code owners September 3, 2024 18:31

alliepiper requested review from wmaxey, ericniebler and miscco September 3, 2024 18:31

alliepiper marked this pull request as draft September 3, 2024 18:31

alliepiper force-pushed the shared_infra branch 2 times, most recently from f8f39ab to c00f93a Compare September 3, 2024 19:43

alliepiper mentioned this pull request Sep 4, 2024

[BUG]: cudax should build with -Wmissing-field-initializers #2357

Closed

1 task

alliepiper force-pushed the shared_infra branch from c00f93a to 841dc46 Compare September 4, 2024 02:45

miscco reviewed Sep 4, 2024

View reviewed changes

jrhemstad assigned alliepiper Sep 4, 2024

alliepiper linked an issue Sep 4, 2024 that may be closed by this pull request

[FEA]: Consolidate shared CMake infrastructure. #2366

Open

1 task

alliepiper force-pushed the shared_infra branch 2 times, most recently from 11cfd31 to fd8841b Compare September 4, 2024 22:16

bernhardmgruber mentioned this pull request Sep 5, 2024

Add -Wmissing-field-initializers to cudax #2373

Merged

alliepiper force-pushed the shared_infra branch from fd8841b to f59b985 Compare September 6, 2024 16:36

alliepiper force-pushed the shared_infra branch 6 times, most recently from ad65e79 to b94a8db Compare September 16, 2024 21:58

alliepiper force-pushed the shared_infra branch 3 times, most recently from 014a013 to 4cc0dd8 Compare September 17, 2024 18:46

alliepiper added 2 commits September 19, 2024 16:12

Migrate CMake packages to central location.

62428fd

alliepiper added 6 commits September 19, 2024 16:12

Migrate install rules to top-level, use cccl_generate_install_rules…

3d4d588

… helper to reduce redundancy.

Remove libcudacxx export tests.

e26b0b0

These tests are redundant with CCCL-level tests, and no longer function now that install rules are centralized at the top level.

Make std arg for test_thrust.ps1 optional.

caef801

Consolidate header testing infra.

26c6adb

Ensure that cudax::Thrust target is only created once.

6c2ebc6

TEMP matrix

26c3bf5

[skip-vdc][skip-docs][skip-rapids]

alliepiper force-pushed the shared_infra branch from 4cc0dd8 to 26c3bf5 Compare September 19, 2024 20:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unify shared infra #2354

Unify shared infra #2354

alliepiper commented Sep 3, 2024

miscco left a comment

github-actions bot commented Sep 17, 2024

🟨 cub: Pass: 81%/11 | Total: 13h 07m | Avg: 1h 11m | Max: 1h 42m | Hits: 99%/727

🟨 libcudacxx: Pass: 60%/5 | Total: 54m 30s | Avg: 10m 54s | Max: 21m 28s | Hits: 99%/2903

🟨 thrust: Pass: 87%/8 | Total: 1h 48m | Avg: 13m 31s | Max: 26m 56s | Hits: 99%/2230

🟩 cudax: Pass: 100%/5 | Total: 27m 52s | Avg: 5m 34s | Max: 11m 32s | Hits: 23%/102

🟩 cccl: Pass: 100%/4 | Total: 19m 20s | Avg: 4m 50s | Max: 5m 43s

🛠️ Override Matrix

👃 Inspect Changes

Modifications in project?

Modifications in project or dependencies?

🏃‍ Runner counts (total jobs: 33)

Unify shared infra #2354

Are you sure you want to change the base?

Unify shared infra #2354

Conversation

alliepiper commented Sep 3, 2024

Description

miscco left a comment

Choose a reason for hiding this comment

github-actions bot commented Sep 17, 2024

🟨 cub: Pass: 81%/11 | Total: 13h 07m | Avg: 1h 11m | Max: 1h 42m | Hits: 99%/727

🟨 libcudacxx: Pass: 60%/5 | Total: 54m 30s | Avg: 10m 54s | Max: 21m 28s | Hits: 99%/2903

🟨 thrust: Pass: 87%/8 | Total: 1h 48m | Avg: 13m 31s | Max: 26m 56s | Hits: 99%/2230

🟩 cudax: Pass: 100%/5 | Total: 27m 52s | Avg: 5m 34s | Max: 11m 32s | Hits: 23%/102

🟩 cccl: Pass: 100%/4 | Total: 19m 20s | Avg: 4m 50s | Max: 5m 43s

🛠️ Override Matrix

👃 Inspect Changes

Modifications in project?

Modifications in project or dependencies?

🏃‍ Runner counts (total jobs: 33)