Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cannot import MimicExplainer #581

Open
Dewei-Wang-xx opened this issue Sep 22, 2023 · 7 comments
Open

cannot import MimicExplainer #581

Dewei-Wang-xx opened this issue Sep 22, 2023 · 7 comments

Comments

@Dewei-Wang-xx
Copy link

I have recently met this issue:

ImportError: cannot import name 'MimicExplainer' from 'interpret.ext.blackbox' (C:\Users\xxx\Anaconda3\lib\site-packages\interpret\ext\blackbox_init_.py)

but I didn't update the library or related ones. Do anyone see this error as well?

@imatiach-msft
Copy link
Collaborator

@Dewei-Wang-xx can you try to run:

pip uninstall interpret
pip uninstall interpret-core
pip install --upgrade interpret-community

and print the output of those two commands? Usually there is some issue with the extension logic in interpret/interpret-core package that causes this.

@Dewei-Wang-xx
Copy link
Author

@imatiach-msft
thanks for your response. I tried your commands but still cannot import the libraries, including TabularExplainer, MimicExplainer, etc.

with pip show interpret-community, it shows

Name: interpret-community
Version: 0.30.0
Summary: Microsoft Interpret Extensions SDK for Python
Home-page: https://github.com/interpretml/interpret-community
Author: Microsoft Corp
Author-email: [email protected]
License: MIT License
Location: c:\users\wang109\anaconda3\envs\ensemble\lib\site-packages
Requires: interpret-core, ml-wrappers, numpy, packaging, pandas, raiutils, scikit-learn, scipy, shap
Required-by:

There are no script except for "init.py" in \Lib\site-packages\interpret\ext\blackbox

@imatiach-msft
Copy link
Collaborator

imatiach-msft commented Sep 25, 2023

@Dewei-Wang-xx it seems there is some issue with the extension system on your particular machine/environment, but without debugging with you I can't tell which package in the environment is causing it. I would recommend to setup a meeting to debug this together. If that is not possible, the simple workaround is to use the interpret_community namespace instead of the interpret.ext.* namespace.
For example, instead of using:

from interpret.ext.blackbox import MimicExplainer

You can just import using:

from interpret_community.mimic import MimicExplainer

Please see the setup.py file for the full mapping of all classes:

https://github.com/interpretml/interpret-community/blob/main/python/setup.py#L90

@imatiach-msft
Copy link
Collaborator

The whole extension system isn't really needed - it's just there to make it look like the interpret-community package is more integrated with the interpret package and the interpretml ecosystem of python packages. But anyone can just import using the original interpret_community namespace instead of using the interpret.ext.* namespace.

@imatiach-msft
Copy link
Collaborator

imatiach-msft commented Sep 25, 2023

Another way this extension system can break is if there is some package version conflict. For example, I ran the command:

pip install --upgrade interpret

And then I see the failures on import telling me that there is a package version conflict and the version of interpret is too new - downgrading via "pip install interpret-core[required]<=0.4.3" fixes this issue (see the very last line of error):

(sh) PS C:\interpret-community> python
Python 3.8.17 | packaged by conda-forge | (default, Jun 16 2023, 07:01:59) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from interpret.ext.blackbox import MimicExplainer
C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\numpy\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:
C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\numpy\.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll
C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\numpy\.libs\libopenblas64__v0.3.21-gcc_10_3_0.dll
  warnings.warn("loaded more than 1 DLL from .libs:"
C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\interpret\ext\extension_utils.py:74: UserWarning: Failure while loading interpret_ext_blackbox. Failed to load entrypoint KernelExplainer = interpret_community.shap:KernelExplainer with exception Traceback (most recent call last):
  File "C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\interpret\ext\extension_utils.py", line 46, in load_class_extensions
    extension_class = entrypoint.load()
  File "C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\pkg_resources\__init__.py", line 2517, in load
    self.require(*args, **kwargs)
  File "C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\pkg_resources\__init__.py", line 2540, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\pkg_resources\__init__.py", line 829, in resolve
    dist = self._resolve_dist(
  File "C:\Users\ilmat\AppData\Local\miniconda3\envs\sh\lib\site-packages\pkg_resources\__init__.py", line 875, in _resolve_dist
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (interpret-core 0.4.4 (c:\users\ilmat\appdata\local\miniconda3\envs\sh\lib\site-packages), Requirement.parse('interpret-core[required]<=0.4.3,>=0.1.20'))

@Dewei-Wang-xx
Copy link
Author

@Dewei-Wang-xx it seems there is some issue with the extension system on your particular machine/environment, but without debugging with you I can't tell which package in the environment is causing it. I would recommend to setup a meeting to debug this together. If that is not possible, the simple workaround is to use the interpret_community namespace instead of the interpret.ext.* namespace. For example, instead of using:

from interpret.ext.blackbox import MimicExplainer

You can just import using:

from interpret_community.mimic import MimicExplainer

Please see the setup.py file for the full mapping of all classes:

https://github.com/interpretml/interpret-community/blob/main/python/setup.py#L90

@imatiach-msft thanks a lot! this works for me.

But how to get the " from interpret.ext.blackbox import MimicExplainer" working?
In previous scripts, I have imported LGBMExplainableModel from interpret.ext.glassbox, TabularExplainer from interpret.ext.blackbox, etc., it may cause some issues to fix them one by one. Thank you!

@imatiach-msft
Copy link
Collaborator

@Dewei-Wang-xx I'm honestly not sure - there should be some error message like above or during install but you are not seeing one. I think the best way to debug this would be to setup a call since you are not seeing anything.

The code to setup the extension system is not in this package but in interpret - specifically this method:

https://github.com/interpretml/interpret/blob/develop/python/interpret-core/interpret/ext/extension_utils.py#L33

I think the best way to debug would be to install interpret-core package locally, call this method here in a jupyter notebook:

https://github.com/interpretml/interpret/blob/develop/python/interpret-core/interpret/ext/glassbox/__init__.py#L8

And then either add print statements or use pdb to debug through that extension_utils method above to see what error is thrown or why those extensions aren't getting registered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants