Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use celery to schedule periodic tasks #27

Merged
merged 19 commits into from
Nov 27, 2023
Merged

Use celery to schedule periodic tasks #27

merged 19 commits into from
Nov 27, 2023

Conversation

Iain-S
Copy link
Contributor

@Iain-S Iain-S commented Nov 22, 2023

Summary

As we add more periodic tasks (e.g. if we add usage forecasting in addition to the existing abolishments and daily emails), switching to a more widely used task management option makes sense.

Celery is a task queue, which requires a broker to mediate between clients and workers and, optionally, a backend for result storage. I have used Redis as the broker as it was easier to install than Rabbit MQ. Since we don't do anything with the results of completed tasks, I have omitted a backend but we could use Redis or Postgres if we need one in the future.

ToDo

  • Fix the error that can occur if we try to insert a daily summary email for subscription UUID(int=0).
  • Add Europe/London timezone?
  • Shut down redis and celery on exit (particularly if using the runtests script.
  • Set up Celery logging (currently logging to file)

@Iain-S Iain-S added WIP Work In Progress (do not merge) and removed WIP Work In Progress (do not merge) labels Nov 22, 2023
.github/workflows/test.yml Show resolved Hide resolved
.github/workflows/test.yml Show resolved Hide resolved
Dockerfile Show resolved Hide resolved
docs/content/logging.md Show resolved Hide resolved
rctab/tasks.py Show resolved Hide resolved
rctab/tasks.py Show resolved Hide resolved
@Iain-S Iain-S added WIP Work In Progress (do not merge) and removed WIP Work In Progress (do not merge) labels Nov 27, 2023
Copy link
Contributor

@joseph-palmer joseph-palmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@Iain-S Iain-S merged commit 21ef960 into main Nov 27, 2023
5 checks passed
@Iain-S Iain-S deleted the celery branch November 27, 2023 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants