Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3.12 regression: cannot create weak reference to 'typing.TypeVar' object #106403

Closed
encukou opened this issue Jul 4, 2023 · 4 comments
Closed
Assignees
Labels
3.12 bugs and security fixes 3.13 bugs and security fixes topic-typing type-bug An unexpected behavior, bug, or error

Comments

@encukou
Copy link
Member

encukou commented Jul 4, 2023

This works in Python 3.11, but fails in 3.12b3:

import weakref
import typing
weakref.ref(typing.TypeVar('T'))

I'm assuming it's an unintentional side effect of the PEP-695 implementation? Unfortunately I don't have time to investigate right now, let me know if I should.

Reported in cloudpipe/cloudpickle#507

Linked PRs

@encukou encukou added the type-bug An unexpected behavior, bug, or error label Jul 4, 2023
@AlexWaygood AlexWaygood added topic-typing 3.12 bugs and security fixes 3.13 bugs and security fixes labels Jul 4, 2023
@JelleZijlstra
Copy link
Member

Fixing.

The new typing.TypeAliasType is also not weakrefable:

>>> type T = int
>>> weakref.ref(T)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot create weak reference to 'typing.TypeAliasType' object

But neither is the existing UnionType:

>>> weakref.ref(int | str)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot create weak reference to 'types.UnionType' object

So maybe that's fine?

@ogrisel
Copy link
Contributor

ogrisel commented Jul 6, 2023

Thanks @JelleZijlstra!

So maybe that's fine?

I suspect that this might cause problems with cloudpickle when executing interactively defined functions with type annotations in their signatures on remote Python workers (e.g. in Dask, Ray or PySpark / databricks).

I did not have time to investigate the details myself yet but @musicinmybrain tried to test your fix in cloudpickle in cloudpipe/cloudpickle#507 (comment) and there are still segfaults to debug.

@JelleZijlstra
Copy link
Member

I merged my PR that re-adds weakref support for TypeVars. I haven't tried to debug the segfaults in the cloudpickle tests, but let me know if those can be minified to something that shows the issue is in CPython.

JelleZijlstra added a commit that referenced this issue Jul 11, 2023
…-106418) (#106635)

gh-106403: Restore weakref support for TypeVar and friends (GH-106418)
(cherry picked from commit 945d3cb)

Co-authored-by: Jelle Zijlstra <[email protected]>
@Yhg1s
Copy link
Member

Yhg1s commented Jul 31, 2023

Closing this as the original report is (believed to be) fixed. If the other cloudpickle issues are caused by bugs in 3.12 it probably deserves its own issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.12 bugs and security fixes 3.13 bugs and security fixes topic-typing type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

5 participants