Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

extension-redis memory leak with extremely frequent doc updates #854

Open
entropi opened this issue Sep 17, 2024 · 0 comments
Open

extension-redis memory leak with extremely frequent doc updates #854

entropi opened this issue Sep 17, 2024 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@entropi
Copy link

entropi commented Sep 17, 2024

Description
When performing a large number of concurrent DirectConnection updates to a handful of documents, a rapidly escalating memory leak was observed. Waiting and forcing garbage collection had no impact on memory growth.

This appears to be separate from (though possibly related to ) #846 as the behavior was observed even when executing rapid updates on only a small number of distinct documents, so it doesn't just appear to be due to not unloading docs.

Steps to reproduce the bug
Steps to reproduce the behavior:

Perform many (hundreds to thousands) of updates (in my case via DirectConnection) to a finite set of documents in a very small window. Observe rapid memory growth, which calls to GC do not clear.

Expected behavior
Memory stays at a reasonable level. setTimeout delays to account for sync remain at one per documentName.

Environment?

  • Hocuspocus version: 2.13.5

Additional context
Lowering the disconnectDelay to 1ms removed the leak (though obviously at the cost of the benefit of the delay for syncs), and raising it to 5000ms during rapid updates escalated the memory growth curve. I suspect that it's related to the fact that we're potentially stacking hundreds or thousands of setTImeout operations on the same document in afterStoreDocument within the disconnectDelay window during rapid edits.

A forthcoming The Linked PR appears to mitigate the issue by clearing any pending timeout operation on a document when a new afterStoreDocument or onDisconnect hook fires for the same documentName, instead of always adding new ones on top.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants