Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revisit the stashing logic in MergeBatcherColumnation #426

Open
antiguru opened this issue Nov 23, 2023 · 0 comments
Open

Revisit the stashing logic in MergeBatcherColumnation #426

antiguru opened this issue Nov 23, 2023 · 0 comments
Assignees

Comments

@antiguru
Copy link
Member

As surfaced recently, the stashing logic to recycle empty buffers in the merge batcher is subtle; it should make sure there are at most 2 empty buffers, which is all it will ever need. The vector-based merge batcher implements this, but the logic is not simple to understand. When implementing the columnation-based merge batcher, I got it wrong, causing the seal function to temporarily retain all empty buffers, which caused an OOM situation.

We should revisit that the implementation maints the invariant that there are at most two empty buffers, and reason about why this is sufficient.

@antiguru antiguru self-assigned this Nov 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant