Rustboard Optional Integration

[vwxyzjn]

Hey, the rustboard integration looks pretty cool, but it also makes it harder to install. Any chance you could make the rust extension optional during installation?

(openrlbenchmark-py3.9) ➜  openrlbenchmark git:(rlops-api) ✗ pip install git+https://github.com/wookayin/expt.git
Collecting git+https://github.com/wookayin/expt.git
  Cloning https://github.com/wookayin/expt.git to /private/var/folders/v4/hnqvx3n10m7f_s2sq_k80pzm0000gn/T/pip-req-build-9v0zi9k1
  Running command git clone --filter=blob:none --quiet https://github.com/wookayin/expt.git /private/var/folders/v4/hnqvx3n10m7f_s2sq_k80pzm0000gn/T/pip-req-build-9v0zi9k1
  Resolved https://github.com/wookayin/expt.git to commit 53929b65f3331692d3e85959012be27ab1453191
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: scipy in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from expt==0.5.0.dev62+g53929b6) (1.6.1)
Requirement already satisfied: multiprocess>=0.70.12 in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from expt==0.5.0.dev62+g53929b6) (0.70.13)
Requirement already satisfied: multiprocessing-utils==0.4 in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from expt==0.5.0.dev62+g53929b6) (0.4)
Requirement already satisfied: typeguard>=2.6.1 in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from expt==0.5.0.dev62+g53929b6) (2.13.3)
Requirement already satisfied: matplotlib>=3.0.0 in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from expt==0.5.0.dev62+g53929b6) (3.5.2)
Requirement already satisfied: numpy>=1.16.5 in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from expt==0.5.0.dev62+g53929b6) (1.23.1)
Requirement already satisfied: typing-extensions>=4.0 in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from expt==0.5.0.dev62+g53929b6) (4.3.0)
Requirement already satisfied: pandas>=1.0 in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from expt==0.5.0.dev62+g53929b6) (1.4.3)
Requirement already satisfied: six in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from multiprocessing-utils==0.4->expt==0.5.0.dev62+g53929b6) (1.16.0)
Requirement already satisfied: cycler>=0.10 in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from matplotlib>=3.0.0->expt==0.5.0.dev62+g53929b6) (0.11.0)
Requirement already satisfied: packaging>=20.0 in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from matplotlib>=3.0.0->expt==0.5.0.dev62+g53929b6) (21.3)
Requirement already satisfied: pyparsing>=2.2.1 in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from matplotlib>=3.0.0->expt==0.5.0.dev62+g53929b6) (3.0.9)
Requirement already satisfied: fonttools>=4.22.0 in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from matplotlib>=3.0.0->expt==0.5.0.dev62+g53929b6) (4.34.4)
Requirement already satisfied: pillow>=6.2.0 in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from matplotlib>=3.0.0->expt==0.5.0.dev62+g53929b6) (9.2.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from matplotlib>=3.0.0->expt==0.5.0.dev62+g53929b6) (1.4.4)
Requirement already satisfied: python-dateutil>=2.7 in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from matplotlib>=3.0.0->expt==0.5.0.dev62+g53929b6) (2.8.2)
Requirement already satisfied: dill>=0.3.5.1 in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from multiprocess>=0.70.12->expt==0.5.0.dev62+g53929b6) (0.3.5.1)
Requirement already satisfied: pytz>=2020.1 in /Users/costahuang/Library/Caches/pypoetry/virtualenvs/openrlbenchmark-bCFsFrj8-py3.9/lib/python3.9/site-packages (from pandas>=1.0->expt==0.5.0.dev62+g53929b6) (2022.1)
Building wheels for collected packages: expt
  Building wheel for expt (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for expt (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [31 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.macosx-12.3-x86_64-cpython-39
      creating build/lib.macosx-12.3-x86_64-cpython-39/expt
      copying expt/plot.py -> build/lib.macosx-12.3-x86_64-cpython-39/expt
      copying expt/path_util.py -> build/lib.macosx-12.3-x86_64-cpython-39/expt
      copying expt/_version.py -> build/lib.macosx-12.3-x86_64-cpython-39/expt
      copying expt/util.py -> build/lib.macosx-12.3-x86_64-cpython-39/expt
      copying expt/data_loader.py -> build/lib.macosx-12.3-x86_64-cpython-39/expt
      copying expt/__init__.py -> build/lib.macosx-12.3-x86_64-cpython-39/expt
      copying expt/data_test.py -> build/lib.macosx-12.3-x86_64-cpython-39/expt
      copying expt/path_util_test.py -> build/lib.macosx-12.3-x86_64-cpython-39/expt
      copying expt/plot_test.py -> build/lib.macosx-12.3-x86_64-cpython-39/expt
      copying expt/data_loader_test.py -> build/lib.macosx-12.3-x86_64-cpython-39/expt
      copying expt/data.py -> build/lib.macosx-12.3-x86_64-cpython-39/expt
      copying expt/colors.py -> build/lib.macosx-12.3-x86_64-cpython-39/expt
      running egg_info
      creating expt.egg-info
      writing expt.egg-info/PKG-INFO
      writing dependency_links to expt.egg-info/dependency_links.txt
      writing requirements to expt.egg-info/requires.txt
      writing top-level names to expt.egg-info/top_level.txt
      writing manifest file 'expt.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      adding license file 'LICENSE'
      writing manifest file 'expt.egg-info/SOURCES.txt'
      running build_ext
      running build_rust
      error: Rust toolchain (cargo, rustc) not found. Please install rust toolchain to build expt with the rust extension. If you would like to build expt without the extension, export EXPT_DISABLE_RUST=1 and try again.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for expt
Failed to build expt
ERROR: Could not build wheels for expt, which is required to install pyproject.toml-based projects

[wookayin]

Yes, I was also feeling the same; actually was thinking about this to make it optional. Maybe we can have a rust extension installed when the [rust] modifier is given (E.g. pip install expt[rust]).

At the moment, you can use the EXPT_DISABLE_RUST environment variable (e.g. EXPT_DISABLE_RUST=1 pip install "expt @ git+https://github.com/wookayin/expt") if you are using the development version. But I know that you don't want to this environment variable in your project configuration or setup. I will make expt be installable without the rust toolchains set up.

[vwxyzjn]

Yeah it’s related to #17. It would be great if environment variable injection is not needed.