Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Performance: Implement Caching #28

Open
matheus23 opened this issue Aug 25, 2023 · 0 comments
Open

Performance: Implement Caching #28

matheus23 opened this issue Aug 25, 2023 · 0 comments

Comments

@matheus23
Copy link
Member

CAR mirror can benefit greatly from a number of caches:

  • A Cid -> bool cache that quickly returns whether a CID is already stored
  • A Cid -> Vec<Cid> cache that returns what further CIDs given block links to

More advanced caches are also possible, e.g. which map (UsizePowerOfTwo, Cid) -> BloomFilter, i.e. a whole subdag to a bloom filter with given size. This would make computing blooms faster. Similarily a Cid -> bool that tells us whether a whole subdag is present, (not only a single block).
@matheus23 matheus23 mentioned this issue Aug 31, 2023
Merged
@matheus23 matheus23 mentioned this issue Jan 2, 2024
Merged
matheus23 added a commit that referenced this issue Jan 2, 2024
@matheus23
fix: Correctly handle raw-codec CIDs/blocks (#37)
b22fb94 
Major things in this PR:
- Correctly handle raw-codec CIDs/blocks, they were previously not transferred due to a bug in `IncrementalDagVerification`.
- Update wnfs-common to 0.1.26 so this crate is compatible with the latest rs-wnfs.
- Introduce another cache, a positive cache for checking if we already have a block. (Work on #28)
- Make sure the main request/response futures are `Send`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@matheus23