[Fixed]-Where should you update Celery settings? On the remote worker or sender?

by

6👍

✅

the django celery settings affects only workers running on the django server itself.

if all your workers are remote workers (the way as i do it), then on the sender side all you need is to put the configuration necessary to submit a task to the task queue.

and all the other settings need to be set on the remote workers.

and for the tasks, on the sender side, all i need to do is to define the task signature like this:

@app.task(name='report_task')
def reportTask(self, link):
    pass

then on the worker side, you need to create a new Celery app with the same name and pointing to the same broker; for other celery settings you need to declare them on the remote workers.

and implement the tasks logic on the remote workeres (you can have different tasks logic per worker as long as they have the same task name and the function arguments)

👤Soufiaane

2👍

CELERY_ACKS_LATE = True belongs to worker. It describes if worker should mark the task ‘acknowledged’ immediately after consuming (before completion) or after completing (late). Both methods have their drawbacks and I think you know what you’re doing.

Of course it would be better to have single configuration file for both parties and use it. For example have the common codebase for entire project and after updating the file in VCS and deploy – restart all parties.

But in this case with this particular flag you can restart only workers.

👤baldr

Leave a comment