[Fixed]-Can celery,celerybeat and django-celery-beat dynamically add/remove tasks in runtime without restart celerybeat?

1👍

This github issue

[You can’t add or delete tasks in celerybeat] currently, you have to restart beat.

No. In order to refresh tasks or task timing inside celery[beat], you must restart the celery[beat] instance. Tasks are loaded into memory at runtime. In order to change/add a task, you must refresh the instance.

You may consider using self recurring tasks, using custom timings and conditional execution. Example:

from datetime import timedelta
from celery import shared_task

@shared_task
def check_conditions():
    # Do some db-level code
    if condition:
        check_conditions.apply_async(eta=timedelta(hours=6))

I use this in production and it performs well.

If you need to reload tasks, just programatically restart celery[beat]:

@shared_task
def autoreload():
    if condition:
        execute_shell_code_to_restart_celery()

I have not used this and cannot vouch for it’s usability, but theoretically should work.

This github issue

I have to reload beat to get this changes updated on worker
… using django-celery-beat …
This problem is still present on 4.0.2 and on master, tested [on December 21st, 2016].

0👍

Warning

If you change the Django TIME_ZONE setting your periodic task schedule will still be based on the old timezone.

To fix that you would have to reset the "last run time" for each periodic task:

 from django_celery_beat.models import PeriodicTask, PeriodicTasks
 PeriodicTask.objects.all().update(last_run_at=None)
 for task in PeriodicTask.objects.all():
     PeriodicTasks.changed(task)

0👍

I was facing the exact same problem as you.
The problem is in the django-celery-beat package which uses DatabaseScheduler and the way its implemented, if you want to dynamically manage periodic tasks (add, delete, update) without restarting the celery beat everytime you add a task and if you are willing to use Redis as well you can use this library https://pypi.org/project/celery-redbeat/ .
If you want more control over your tasks I’ve written a library based on it as well and works natively with Django https://pypi.org/project/django-redbeat/

Leave a comment