Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High memory usage when processing large volume of tasks #23

Open
seocam opened this issue Sep 23, 2022 · 0 comments
Open

High memory usage when processing large volume of tasks #23

seocam opened this issue Sep 23, 2022 · 0 comments

Comments

@seocam
Copy link
Member

seocam commented Sep 23, 2022

In some periods of the day we receive from 300.000 to 500.000 tasks at once. This is causing the memory usage of celery-logger to increase until it's killed. The memory usage never reduces even after the messages are processed (every drop in chart bellow was caused by a kill).

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant