[Solved]-Celery chaining tasks sequentially

17👍

A chain is always passed the previous result as a first argument. From the chains documentation:

The linked task will be applied with the result of its parent task as the first argument, which in the above case will result in mul(4, 16) since the result is 4.

Your upload_ftp_image task doesn’t accept this extra argument, and thus it fails.

You have a fine use case for chaining; the second task is guaranteed to be called after the first task is completed (otherwise the result could not be passed on anyway).

Simply add an argument for the result from the previous task:

def upload_ftp_image(download_result, ftp_server, username , password , file , directory):

You could make some use of that result value; maybe make it the download method return the path of the downloaded file so the upload method knows what to upload?

25👍

Another option if you don’t want the return value of the previous task to be used as an argument, is to use ‘immutability’.

http://docs.celeryproject.org/en/latest/userguide/canvas.html#immutability

Instead of defining your subtasks as:

download_ftp_image.s(...) and upload_ftp_image.s(...)

define them as:

download_ftp_image.si(...) and upload_ftp_image.si(...)

And you can now use the tasks with the usual number of arguments in a chain.

👤mpaf

Leave a comment