Fix default cache dataloader raise key error on non-existing key #3569

Dartt0n · 2024-07-11T16:04:22Z

Description

Right now, calling the .clean(key) method on a non-cached key will raise a KeyError exception. There are several ways to solve this problem:

Create a custom CustomDefaultCache class and override the delete method to first check if the key exists.
Check if the key is cached before deleting it each time the .clean method is called.
Cache the key before calling .clean.
Set the default behavior to be exception-free.
Among these options, the fourth solution seems to be the best, so these changes are proposed.

Types of Changes

Issues Fixed or Closed by This PR

DataLoader with a DefaultCache throws key error when attempting to delete key that does not exist #2464

Checklist

My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have read the CONTRIBUTING document.
I have added tests to cover my changes.
I have tested the changes and verified that they work and don't break anything (as well as I can manage).

Summary by Sourcery

This pull request fixes a bug where calling .clean(key) on a non-cached key in the default dataloader would raise a KeyError. The fix ensures that the method no longer throws an exception in such cases. Documentation and tests have been updated accordingly.

Bug Fixes:
- Fixed issue where calling .clean(key) on a non-cached key in the default dataloader would raise a KeyError.
Documentation:
- Added a note in RELEASE.md to document the change in behavior for the .clean(key) method in the default dataloader.
Tests:
- Added a test case to ensure that calling .clean(key) on a non-cached key does not raise an error.

Closes strawberry-graphql#2464

sourcery-ai · 2024-07-11T16:04:29Z

Reviewer's Guide by Sourcery

This pull request addresses a bug where calling the .clean(key) method on a non-cached key would raise a KeyError. The delete method in strawberry/dataloader.py was modified to use pop with a default value to avoid this error. Corresponding test cases were added to tests/test_dataloaders.py to ensure the new behavior is correctly implemented. Additionally, release notes were added in RELEASE.md to document this patch change.

File-Level Changes

Files	Changes
`strawberry/dataloader.py` `tests/test_dataloaders.py`	Updated the `delete` method in `dataloader.py` to prevent `KeyError` and added corresponding test cases in `test_dataloaders.py`.

Tips

Trigger a new Sourcery review by commenting @sourcery-ai review on the pull request.
Continue your discussion with Sourcery by replying directly to review comments.
You can change your review settings at any time by accessing your dashboard:
- Enable or disable the Sourcery-generated pull request summary or reviewer's guide;
- Change the review language;
You can always contact us if you have any questions or feedback.

for more information, see https://pre-commit.ci

sourcery-ai

Hey @Dartt0n - I've reviewed your changes and they look great!

Here's what I looked at during the review

🟢 General issues: all looks good
🟢 Security: all looks good
🟡 Testing: 1 issue found
🟢 Complexity: all looks good
🟡 Documentation: 3 issues found

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment to tell me if it was helpful.}

tests/test_dataloaders.py

sourcery-ai · 2024-07-11T16:05:22Z

RELEASE.md

@@ -0,0 +1,12 @@
+Release type: patch
+
+Calling `.clean(key)` on default dataloader with non-existing `key` will not throw `KeyError` error anymore. Example:


issue (documentation): Redundant phrase 'KeyError error'

Consider changing 'KeyError error' to just 'KeyError' to avoid redundancy.

sourcery-ai · 2024-07-11T16:05:22Z

RELEASE.md

+async def load_data(keys):
+    return [str(key) for key in keys]
+
+dataloader = DataLoader(load_fn=load_data)


suggestion (documentation): Mention import statement for DataLoader

Consider adding a comment or a line to indicate that DataLoader should be imported from the relevant module to avoid confusion.

Suggested change

dataloader = DataLoader(load_fn=load_data)

```suggestion

```python

from some_module import DataLoader

dataloader = DataLoader(load_fn=load_data)

sourcery-ai · 2024-07-11T16:05:22Z

RELEASE.md

+dataloader.clean(42) # does not throw KeyError anymore
+```
+
+This is a patch release, so no breaking changes.


suggestion (documentation): Improve sentence flow

Consider rephrasing to 'This is a patch release with no breaking changes.'

Suggested change

This is a patch release, so no breaking changes.

This is a patch release with no breaking changes.

botberry · 2024-07-11T16:06:17Z

Thanks for adding the RELEASE.md file!

Here's a preview of the changelog:

Calling .clean(key) on default dataloader with non-existing key will not throw KeyError anymore. Example:

from strawberry.dataloader import DataLoader


async def load_data(keys):
    return [str(key) for key in keys]


dataloader = DataLoader(load_fn=load_data)
dataloader.clean(42)  # does not throw KeyError anymore

This is a patch release with no breaking changes.

Here's the tweet text:

🆕 Release (next) is out! Thanks to Anton Kudryavtsev for the PR 👏

Get it here 👉 https://strawberry.rocks/release/(next)

botberry · 2024-07-11T16:06:26Z

Thanks for adding the RELEASE.md file!

Here's a preview of the changelog:

Calling .clean(key) on default dataloader with non-existing key will not throw KeyError error anymore. Example:

async def load_data(keys):
    return [str(key) for key in keys]


dataloader = DataLoader(load_fn=load_data)
dataloader.clean(42)  # does not throw KeyError anymore

This is a patch release, so no breaking changes.

Here's the tweet text:

🆕 Release (next) is out! Thanks to Anton Kudryavtsev for the PR 👏

Get it here 👉 https://strawberry.rocks/release/(next)

codecov · 2024-07-11T16:11:49Z

Codecov Report

Attention: Patch coverage is 83.33333% with 1 line in your changes missing coverage. Please review.

Project coverage is 94.34%. Comparing base (bea5cac) to head (5b87885).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3569      +/-   ##
==========================================
- Coverage   96.58%   94.34%   -2.25%     
==========================================
  Files         524      520       -4     
  Lines       33632    32486    -1146     
  Branches     5577     3721    -1856     
==========================================
- Hits        32485    30650    -1835     
- Misses        912     1549     +637     
- Partials      235      287      +52

codspeed-hq · 2024-07-11T16:11:51Z

CodSpeed Performance Report

Merging #3569 will not alter performance

_{Comparing Dartt0n:main (5b87885) with main (bea5cac)}

Summary

✅ 13 untouched benchmarks

for more information, see https://pre-commit.ci

Dartt0n · 2024-07-11T20:05:28Z

Hmm, seems like type linting fails to resolve:

strawberry/dataloader.py:87: error: No overload variant of "pop" of "dict" matches argument types "Hashable", "None"  [call-overload]

However, a overload exist:

strawberry/dataloader.py:87: note: Possible overload variants:
strawberry/dataloader.py:87: note:     def pop(self, Hashable, /) -> Future[T]
strawberry/dataloader.py:87: note:     def pop(self, Hashable, Future[T], /) -> Future[T]
strawberry/dataloader.py:87: note:     def [_T] pop(self, Hashable, _T, /) -> Future[T] | _T # this one

The type is def [None] pop(self, Hashable, NoneType, /) -> Future[T] | None

Will change code to

if key in self.cache_map:
    del self.cache_map[key]

to satisfy linter

Dartt0n · 2024-07-11T20:19:36Z

Seems thats all check passed, waiting for the review

patrick91 · 2024-07-14T15:05:48Z

@Dartt0n in what cases would this be useful? 😊

wouldn't an error be a useful indication that something was wrong somewhere else?

Dartt0n · 2024-07-14T19:55:10Z

@Dartt0n in what cases would this be useful? 😊

wouldn't an error be a useful indication that something was wrong somewhere else?

For example, some services modify an object, and this service needs to invalidate the cache for the modified object. However, the service does not know whether the object has already been loaded into the cache. This is a common problem for larger applications.

I think most developers who face this issue either use try-catch blocks or create their own wrapper (like original issue suggest). Therefore, I believe that making this behavior non-exceptional would be beneficial for the developer's user experience.

Additionally, as an example, the Redis DEL command does not throw an error when invalidating a non-existent cache entry.
https://redis.io/docs/latest/commands/del/

If you think that this explicit behavior would be more appropriate for the project, I suggest including a clear statement about it in the documentation in the "Cache Invalidation" section.
https://strawberry.rocks/docs/guides/dataloaders#cache-invalidation

Dartt0n added 2 commits July 11, 2024 18:53

Fix default cache dataloader raise key error on non-existing key

724ebe3

Closes strawberry-graphql#2464

Modify test to test behaviour on non-existing cache key

7a2a88a

[pre-commit.ci] auto fixes from pre-commit.com hooks

5fe7f6f

for more information, see https://pre-commit.ci

sourcery-ai bot reviewed Jul 11, 2024

View reviewed changes

botberry added bot:has-release-file bot:release-type-patch labels Jul 11, 2024

Dartt0n and others added 3 commits July 11, 2024 19:13

Update test to explicitly check no exception was raised

92a4ff0

Update RELEASE.md

eab45a5

[pre-commit.ci] auto fixes from pre-commit.com hooks

4ac166d

for more information, see https://pre-commit.ci

Dartt0n added 2 commits July 11, 2024 23:10

Modify test to test behaviour on non-existing cache key

c189236

Address B011 Do not assert False

5b87885

botberry removed the bot:release-type-patch label Sep 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix default cache dataloader raise key error on non-existing key #3569

Fix default cache dataloader raise key error on non-existing key #3569

Dartt0n commented Jul 11, 2024 •

edited by sourcery-ai bot

Loading

sourcery-ai bot commented Jul 11, 2024 •

edited

Loading

sourcery-ai bot left a comment

sourcery-ai bot Jul 11, 2024

sourcery-ai bot Jul 11, 2024

sourcery-ai bot Jul 11, 2024

botberry commented Jul 11, 2024 •

edited

Loading

botberry commented Jul 11, 2024

codecov bot commented Jul 11, 2024 •

edited

Loading

codspeed-hq bot commented Jul 11, 2024 •

edited

Loading

Dartt0n commented Jul 11, 2024

Dartt0n commented Jul 11, 2024

patrick91 commented Jul 14, 2024

Dartt0n commented Jul 14, 2024

		@@ -0,0 +1,12 @@
		Release type: patch

		Calling `.clean(key)` on default dataloader with non-existing `key` will not throw `KeyError` error anymore. Example:

-dataloader = DataLoader(load_fn=load_data)
+```suggestion
+```python
+from some_module import DataLoader
+dataloader = DataLoader(load_fn=load_data)

Fix default cache dataloader raise key error on non-existing key #3569

Are you sure you want to change the base?

Fix default cache dataloader raise key error on non-existing key #3569

Conversation

Dartt0n commented Jul 11, 2024 • edited by sourcery-ai bot Loading

Description

Types of Changes

Issues Fixed or Closed by This PR

Checklist

Summary by Sourcery

sourcery-ai bot commented Jul 11, 2024 • edited Loading

Reviewer's Guide by Sourcery

File-Level Changes

sourcery-ai bot left a comment

Choose a reason for hiding this comment

sourcery-ai bot Jul 11, 2024

Choose a reason for hiding this comment

sourcery-ai bot Jul 11, 2024

Choose a reason for hiding this comment

sourcery-ai bot Jul 11, 2024

Choose a reason for hiding this comment

botberry commented Jul 11, 2024 • edited Loading

botberry commented Jul 11, 2024

codecov bot commented Jul 11, 2024 • edited Loading

Codecov Report

codspeed-hq bot commented Jul 11, 2024 • edited Loading

CodSpeed Performance Report

Merging #3569 will not alter performance

Summary

Dartt0n commented Jul 11, 2024

Dartt0n commented Jul 11, 2024

patrick91 commented Jul 14, 2024

Dartt0n commented Jul 14, 2024

Dartt0n commented Jul 11, 2024 •

edited by sourcery-ai bot

Loading

sourcery-ai bot commented Jul 11, 2024 •

edited

Loading

botberry commented Jul 11, 2024 •

edited

Loading

codecov bot commented Jul 11, 2024 •

edited

Loading

codspeed-hq bot commented Jul 11, 2024 •

edited

Loading