Merge branch 'main' into enh/offset-device-scan

NVIDIA · Aug 16, 2024 · 4aaac08 · 4aaac08
2 parents 994c62c + ba9e9bb
commit 4aaac08
Show file tree

Hide file tree

Showing 262 changed files with 16,982 additions and 13,278 deletions.
diff --git a/.devcontainer/README.md b/.devcontainer/README.md
@@ -1,26 +1,32 @@
 > **Note**
-> The instructions in this README are specific to Linux development environments. Instructions for Windows are coming soon!
+> The instructions in this README are specific to Linux development environments (including WSL on Windows). Instructions for native Windows development (e.g., `msvc`) are coming soon!
 
 [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/NVIDIA/cccl?quickstart=1&devcontainer_path=.devcontainer%2Fdevcontainer.json)
 
 # CCCL Dev Containers
 
-CCCL uses [Development Containers](https://containers.dev/) to provide consistent and convenient development environments for both local development and for CI. This guide covers setup in [Visual Studio Code](#quickstart-vscode-recommended) and [Docker](#quickstart-docker-manual-approach). The guide also provides additional instructions in case you want use WSL.
+CCCL uses [Dev Containers](https://containers.dev/) to provide consistent and convenient development environments for both local development and for CI.
 
-## Table of Contents
-1. [Quickstart: VSCode (Recommended)](#vscode)
-2. [Quickstart: Docker (Manual Approach)](#docker)
-3. [Quickstart: Using WSL](#wsl)
+VSCode offers the most convenient experience with Dev Containers due to its tight native integration, however, our containers are also fully usable without VSCode by leveraging Docker directly.
 
-## Quickstart: VSCode (Recommended) <a name="vscode"></a>
+## Table of Contents
+1. [Quickstart: VSCode on Linux (Recommended)](#vscode)
+2. [Quickstart: VSCode on WSL (Recommended for Windows)](#wsl)
+3. [Quickstart: Docker on Linux (Manual Approach)](#docker)
 
+## Quickstart: VSCode on Linux (Recommended) <a name="vscode"></a>
 ### Prerequisites
 - [Visual Studio Code](https://code.visualstudio.com/)
 - [Remote - Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)
 - [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
 - [Docker](https://docs.docker.com/engine/install/) - This is only for completeness because it should already be implicitly installed by the Dev Containers extension
 
-### Steps
+#### GPU Prerequisites (only needed for executing tests that require a GPU)
+- Supported NVIDIA GPU
+- [NVIDIA Driver](https://www.nvidia.com/Download/index.aspx?lang=en-us)
+- [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
+
+### Steps <a name="vscode-devcontainer-steps"></a>
 
 1. Clone the Repository
     ```bash
@@ -32,7 +38,7 @@ CCCL uses [Development Containers](https://containers.dev/) to provide consisten
 
    ![Shows "Reopen in Container" prompt when opening the cccl directory in VScode.](./img/reopen_in_container.png)
 
-   - Alternatively, use the Command Palette to start a Dev Container. Press `Ctrl+Shift+P` to open the Command Palette. Type "Remote-Containers: Reopen in Container" and select it.
+   - Alternatively, use `ctrl+shift+p` to open the Command Palette and type "Remote-Containers: Reopen in Container" and select it.
 
      ![Shows "Reopen in Container" in command pallete.](./img/open_in_container_manual.png)
 
@@ -42,11 +48,14 @@ CCCL uses [Development Containers](https://containers.dev/) to provide consisten
 
 5. VSCode will initialize the selected Dev Container. This can take a few minutes the first time.
 
-6. Once initialized, the local `cccl/` directory is mirrored into the container to ensure any changes are persistent.
+6. (Optional) Authenticate with GitHub
+   - After container startup, you will be asked if you would like to authenticate with GitHub. This is for access to CCCL's distributed `sccache` storage. If you are not an NVIDIA employee, you can safely ignore this step. For more information, see the [`sccache`](#sccache) section below.
 
-7. Done! See the [contributing guide](../CONTRIBUTING.md#building-and-testing) for instructions on how to build and run tests.
+7. Once initialized, the local `cccl/` directory is mirrored into the container to ensure any changes are persistent.
 
-### (Optional) Authenticate with GitHub for `sccache`
+8. Done! See the [contributing guide](../CONTRIBUTING.md#building-and-testing) for instructions on how to build and run tests.
+
+### (Optional) Authenticate with GitHub for `sccache` <a name="sccache"></a>
 
 After starting the container, there will be a prompt to authenticate with GitHub. This grants access to a [`sccache`](https://github.com/mozilla/sccache) server shared with CI and greatly accelerates local build times. This is currently limited to NVIDIA employees belonging to the `NVIDIA` or `rapidsai` GitHub organizations.
 
@@ -60,11 +69,110 @@ To manually trigger this authentication, execute the `devcontainer-utils-vault-s
 
 For more information about the sccache configuration and authentication, see the documentation at [`rapidsai/devcontainers`](https://github.com/rapidsai/devcontainers/blob/branch-23.10/USAGE.md#build-caching-with-sccache).
 
+## Quickstart: VSCode on WSL (Recommended for Windows) <a name="wsl"></a>
+
+Windows Subsystem for Linux (WSL) enables you to run a Linux environment directly in Windows.
+This isn't for native Windows development (e.g., compiling with `msvc`), but effectively a more convenient option than setting up a dual-boot Linux/Windows machine.
+Apart from the initial setup of WSL, the process for using CCCL's Dev Containers in WSL is effectively the same as the instructions for Linux, because WSL _is_ Linux.
+
+### Prerequisites
+- Windows OS that supports WSL 2 (Windows 11 or newer)
+- [Windows Subsystem for Linux v2 (WSL 2)](https://learn.microsoft.com/en-us/windows/wsl/install)
+- [Visual Studio Code](https://code.visualstudio.com/) (installed on Windows host)
+- [VSCode Remote Development Extension Pack](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack) (installed on Windows host)
+    - Includes [Dev Containers](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) and [WSL](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-wsl) extensions
+- [Docker](https://docs.docker.com/engine/install/) - (Will be installed automatically by the Remote Development extension)
+
+#### GPU Prerequisites (only needed for executing tests that require a GPU)
+- Supported NVIDIA GPU
+- [NVIDIA Driver](https://www.nvidia.com/Download/index.aspx?lang=en-us) (installed on Windows host)
+- [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) (**installed inside WSL**)
+
+For more details see the official NVIDIA [Getting Started with CUDA on WSL guide](https://docs.nvidia.com/cuda/wsl-user-guide/index.html#getting-started-with-cuda-on-wsl-2).
+
+### Install WSL on your Windows host
+Refer to [Microsoft's documentation](https://learn.microsoft.com/en-us/windows/wsl/install) for the full instructions to install WSL2.
+
+<details>
+  <summary>Click here for the TL;DR version</summary>
+1. Run `Powershell` as an administrator
+![image](https://github.com/user-attachments/assets/2c985887-ca6c-46bc-9e1b-f235ccfd8513)
+
+2. Install WSL 2 by running:
+```bash
+> wsl --install
+```
+3. Restart your computer
+4. If this is your first time installing WSL, upon restarting, it will prompt you to create a username/password to use inside WSL.
+5. Verify `wsl` was succesfully installed by opening Powershell again and run
+```bash
+> wsl -l -v
+  NAME      STATE           VERSION
+* Ubuntu    Running         2
+```
+5. Launch `wsl` and verify your Linux environment
+```
+# In Powershell, start WSL, which will drop you into a terminal session running in Linux
+> wsl
+
+# In the new terminal session, verify your Linux environment by changing to your home directory
+# and displaying the current directory. This should show `/home/*YOUR USER NAME*`
+> cd ~
+> pwd
+/home/jhemstad
+```
+
+Congratulations! You now have WSL installed and can use it as you would a normal Ubuntu/Linux installation.
+This is sufficient for *building* CCCL's tests, if you have a GPU on your system and you would like to use it to run the tests, continue below:
+
+6. (Optional) Install `nvidia-container-toolkit`
+See [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#installing-with-apt) for full instructions.
+
+**Important:** `nvidia-container-toolkit` needs to be installed inside WSL (not on the Windows host). The following commands should be run within the Linux environment.
+
+```bash
+$ curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
+  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
+    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
+    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
+$ sudo apt-get update
+$ sudo apt-get install -y nvidia-container-toolkit
+```
+
+Then configure Docker to use the `nvidia-container-toolkit`:
+```bash
+$ sudo nvidia-ctk runtime configure --runtime=docker
+$ sudo systemctl restart docker
+```
+
+7. (Optional) Verify your GPU is available inside WSL
+Use `nvidia-smi` inside of WSL to verify that your GPU is correctly configured and available from inside the container.
+If not, verify that the NVIDIA GPU driver is correctly installed on your Windows host and `nvidia-container-toolkit` was successfully installed inside of WSL.
+```bash
+$ nvidia-smi
+```
+</details>
+
+### Connect VSCode to WSL
+1. Launch VSCode on your Windows host
+
+2. Connect VSCode to your WSL instance
+- Enter `ctrl + shift + p` to open the command prompt and type "WSL" and click "WSL: Connect to WSL"
+    - If you don't see this option, you need to install the [WSL VSCode Extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-wsl) (comes with the [Remote Development pack ](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack))
+![image](https://github.com/user-attachments/assets/3e0e6af7-4251-4ce9-9204-589ad7daa12a)
+    - To verify VSCode is connected to WSL, you should see the following in the bottom left corner: ![Shows the WSL: Ubuntu status for a successful connection to WSL.](https://github.com/user-attachments/assets/26dbba61-cc96-4ac3-8200-fdb26a8e4a4b)
+
+3. VSCode is now attached to WSL and it is equivalent to running in a native Linux environment. You can now proceed as described in the [section above](#vscode-devcontainer-steps).
+
 ## Quickstart: Docker (Manual Approach) <a name="docker"></a>
 
 ### Prerequisites
+- [Docker](https://docs.docker.com/engine/install/)
+
+#### GPU Prerequisites (only needed for executing tests that require a GPU)
+- Supported NVIDIA GPU
+- [NVIDIA Driver](https://www.nvidia.com/Download/index.aspx?lang=en-us)
 - [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
-- [Docker](https://docs.docker.com/desktop/install/linux-install/)
 
 ### Steps
 1. Clone the repository and use the [`launch.sh`](./launch.sh) script to launch the default container environment
@@ -122,78 +230,3 @@ Click the badge above or [click here](https://codespaces.new/NVIDIA/cccl?quickst
 For more information, see the `.devcontainer/make_devcontainers.sh --help` message.
 
 **Note**: When adding or updating supported environments, modify `matrix.yaml` and then rerun this script to synchronize the `devcontainer` configurations.
-
-## Quickstart: Using WSL <a name="wsl"></a>
-
-> [!NOTE]
-> _Make sure you have the Nvidia driver installed on your Windows host before moving further_. Type in `nvidia-smi` for verification.
-
-### Install WSL on your Windows host
-
-> [!WARNING]
-> Disclaimer: This guide was developed for WSL 2 on Windows 11.
-
-1. Launch a Windows terminal (_e.g. Powershell_) as an administrator.
-
-2. Install WSL 2 by running:
-```bash
-wsl --install
-```
-This should probably install Ubuntu distro as a default.
-
-3. Restart your computer and run `wsl -l -v` on a Windows terminal to verify installation.
-
-<h3 id="prereqs"> Install prerequisites and VS Code extensions</h3>
-
-4. Launch your WSL/Ubuntu terminal by running `wsl` in Powershell.
-
-5. Install the [WSL extension](ms-vscode-remote.remote-wsl) on VS Code.
-
-    - `Ctrl + Shift + P` and select `WSL: Connect to WSL` (it will prompt you to install the WSL extension).
-
-    - Make sure you are connected to WSL with VS Code by checking the bottom left corner of the VS Code window (should indicate "WSL: Ubuntu" in our case).
-
-6. Install the [Dev Containers extension](ms-vscode-remote.remote-containers) on VS Code.
-
-    - In a vanilla system you should be prompted to install `Docker` at this point, accept it. If it hangs you might have to restart VS Code after that.
-
-7. Install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). **Make sure you install the WSL 2 version and not the native Linux one**. This builds on top of Docker so make sure you have Docker properly installed (run `docker --version`).
-
-8. Open `/etc/docker/daemon.json` from within your WSL system (if the file does not exist, create it) and add the following:
-
-```json
-{
-    "runtimes": {
-        "nvidia": {
-            "path": "nvidia-container-runtime",
-            "runtimeArgs": []
-        }
-    }
-}
-```
-
-then run `sudo systemctl restart docker.service`.
-
----
-### Build CCCL in WSL using Dev Containers
-
-9. Still on your WSL terminal run `git clone https://github.com/NVIDIA/cccl.git`
-
-
-10. Open the CCCL cloned repo in VS Code ( `Ctrl + Shift + P `, select `File: Open Folder...` and select the path where your CCCL clone is located).
-
-11. If prompted, choose `Reopen in Container`.
-
-    - If you are not prompted just type `Ctrl + Shift + P` and `Dev Containers: Open Folder in Container ...`.
-
-12. Verify that Dev Container was configured properly by running `nvidia-smi` in your Dev Container terminal. For a proper configuration it is important for the steps in [Install prerequisites and VS Code extensions](#prereqs) to be followed in a precise order.
-
-From that point on, the guide aligns with our [existing Dev Containers native Linux guide](https://github.com/NVIDIA/cccl/blob/main/.devcontainer/README.md) with just one minor potential alteration:
-
-13. If WSL was launched without the X-server enabled, when asked to "authenticate Git with your Github credentials", if you answer **Yes**, the browser might not open automatically, with the following error message.
-
-> Failed opening a web browser at https://github.com/login/device
-  exec: "xdg-open,x-www-browser,www-browser,wslview": executable file not found in $PATH
-  Please try entering the URL in your browser manually
-
-In that case type in the address manually in your web browser https://github.com/login/device and fill in the one-time code.
diff --git a/.devcontainer/cuda11.1-gcc6/devcontainer.json b/.devcontainer/cuda11.1-gcc6/devcontainer.json
@@ -7,7 +7,8 @@
   "initializeCommand": [
     "/bin/bash",
     "-c",
-    "mkdir -m 0755 -p ${localWorkspaceFolder}/.{aws,cache,config}"
+    "mkdir -m 0755 -p ${localWorkspaceFolder}/.{aws,cache,config}; mkdir -m 0755 -p ${localWorkspaceFolder}/build;",
+    "if [[ -n ${WSLENV+set} ]]; then docker volume create cccl-build; else docker volume create --driver local --opt type=none --opt device=${localWorkspaceFolder}/build --opt o=bind cccl-build fi;"
   ],
   "containerEnv": {
     "SCCACHE_REGION": "us-east-2",
@@ -25,7 +26,8 @@
   "mounts": [
     "source=${localWorkspaceFolder}/.aws,target=/home/coder/.aws,type=bind,consistency=consistent",
     "source=${localWorkspaceFolder}/.cache,target=/home/coder/.cache,type=bind,consistency=consistent",
-    "source=${localWorkspaceFolder}/.config,target=/home/coder/.config,type=bind,consistency=consistent"
+    "source=${localWorkspaceFolder}/.config,target=/home/coder/.config,type=bind,consistency=consistent",
+    "source=cccl-build,target=/home/coder/cccl/build"
   ],
   "customizations": {
     "vscode": {
@@ -38,7 +40,7 @@
       "settings": {
         "editor.defaultFormatter": "xaver.clang-format",
         "editor.formatOnSave": true,
-        "clang-format.executable": "/usr/local/bin/clang-format",
+        "clang-format.executable": "/usr/bin/clang-format",
         "clangd.arguments": [
           "--compile-commands-dir=${workspaceFolder}"
         ],

diff --git a/.devcontainer/cuda11.1-gcc7/devcontainer.json b/.devcontainer/cuda11.1-gcc7/devcontainer.json
@@ -7,7 +7,8 @@
   "initializeCommand": [
     "/bin/bash",
     "-c",
-    "mkdir -m 0755 -p ${localWorkspaceFolder}/.{aws,cache,config}"
+    "mkdir -m 0755 -p ${localWorkspaceFolder}/.{aws,cache,config}; mkdir -m 0755 -p ${localWorkspaceFolder}/build;",
+    "if [[ -n ${WSLENV+set} ]]; then docker volume create cccl-build; else docker volume create --driver local --opt type=none --opt device=${localWorkspaceFolder}/build --opt o=bind cccl-build fi;"
   ],
   "containerEnv": {
     "SCCACHE_REGION": "us-east-2",
@@ -25,7 +26,8 @@
   "mounts": [
     "source=${localWorkspaceFolder}/.aws,target=/home/coder/.aws,type=bind,consistency=consistent",
     "source=${localWorkspaceFolder}/.cache,target=/home/coder/.cache,type=bind,consistency=consistent",
-    "source=${localWorkspaceFolder}/.config,target=/home/coder/.config,type=bind,consistency=consistent"
+    "source=${localWorkspaceFolder}/.config,target=/home/coder/.config,type=bind,consistency=consistent",
+    "source=cccl-build,target=/home/coder/cccl/build"
   ],
   "customizations": {
     "vscode": {
@@ -38,7 +40,7 @@
       "settings": {
         "editor.defaultFormatter": "xaver.clang-format",
         "editor.formatOnSave": true,
-        "clang-format.executable": "/usr/local/bin/clang-format",
+        "clang-format.executable": "/usr/bin/clang-format",
         "clangd.arguments": [
           "--compile-commands-dir=${workspaceFolder}"
         ],

diff --git a/.devcontainer/cuda11.1-gcc8/devcontainer.json b/.devcontainer/cuda11.1-gcc8/devcontainer.json
@@ -7,7 +7,8 @@
   "initializeCommand": [
     "/bin/bash",
     "-c",
-    "mkdir -m 0755 -p ${localWorkspaceFolder}/.{aws,cache,config}"
+    "mkdir -m 0755 -p ${localWorkspaceFolder}/.{aws,cache,config}; mkdir -m 0755 -p ${localWorkspaceFolder}/build;",
+    "if [[ -n ${WSLENV+set} ]]; then docker volume create cccl-build; else docker volume create --driver local --opt type=none --opt device=${localWorkspaceFolder}/build --opt o=bind cccl-build fi;"
   ],
   "containerEnv": {
     "SCCACHE_REGION": "us-east-2",
@@ -25,7 +26,8 @@
   "mounts": [
     "source=${localWorkspaceFolder}/.aws,target=/home/coder/.aws,type=bind,consistency=consistent",
     "source=${localWorkspaceFolder}/.cache,target=/home/coder/.cache,type=bind,consistency=consistent",
-    "source=${localWorkspaceFolder}/.config,target=/home/coder/.config,type=bind,consistency=consistent"
+    "source=${localWorkspaceFolder}/.config,target=/home/coder/.config,type=bind,consistency=consistent",
+    "source=cccl-build,target=/home/coder/cccl/build"
   ],
   "customizations": {
     "vscode": {
@@ -38,7 +40,7 @@
       "settings": {
         "editor.defaultFormatter": "xaver.clang-format",
         "editor.formatOnSave": true,
-        "clang-format.executable": "/usr/local/bin/clang-format",
+        "clang-format.executable": "/usr/bin/clang-format",
         "clangd.arguments": [
           "--compile-commands-dir=${workspaceFolder}"
         ],