The diagram below outlines the task life-cycle. Transitions drawn by solid black lines are initiated by workers. While dashes transitions are initiated at the initiative of the queue, or its consumers.
When a task is created it is unscheduled until (i) it is scheduled by
queue.scheduleTask, or (ii) all of its dependencies have been
satisfied. Notice that if a task doesn't have any dependencies, it will
transition from unscheduled to pending as soon as it is created.
If creating a task that is to be scheduled later by some external service, the
task can be given a self-dependency, which will cause it to remain unscheduled
scheduleTask is called, or (b)
task.deadline is exceeded.
By default a task becomes pending when all dependencies in
task.dependencies have been resolved completed. These semantics can be
tweaked by setting
task.requires = 'all-resolved', which causes the task to
become pending when all dependencies have been resolved as either completed,
failed, or exception. If you need semantics other than
all-resolved, you can implement that using an intermediary decision task with
When a task becomes pending it can be claimed by a worker that wishes to complete the task. Once claimed the task atomically transitions to the running state.
A task can become pending more than once, e.g. if the worker crashes while processing the task. To track this a task may have one or more runs. A task without any runs is unscheduled, a task with pending run is said to pending.
Runs are number starting from zero, and only the last run can be pending or running, ensuring that two workers aren't working on the same task at the same time.
If the task isn't claimed by a worker before
task.deadline the pending run,
and by implication the task, will be resolved as exception. The same happens
if the task is canceled.
The queue exposes an approximate number of pending tasks for each
workerType combination, for use by provisioners that are able
to dynamically scale up the number of workers.
When a pending task is claimed by a worker it becomes running. More accurately it is the latest run from the task that is claimed by a worker, and hence, transitioning the state of the run/task from pending to running.
While a task/run is pending it must be repeatedly reclaimed by the worker.
This is indicated by the
takenUntil property. If not reclaimed before
takenUntil the run is resolved as exception, and a if retries aren't
exhausted a new pending run will be added to task, rendering the task
pending again. This ensures that tasks will be retried if workers disappear.
When claiming and reclaiming tasks the worker will receive temporary credentials
from the queue. These can be used to (i) upload artifacts, (ii) reclaim the
task/run, and, (iii) resolve the task. Finally, these temporary credentials also
task.scopes, allowing the worker to use any scope granted to the task.
Once execution of a task is finished and all artifacts/logs have been uploaded,
the worker processing the task will transition the task/run to completed or
Once a task enters this state it is resolved, and will remain stable until its
expiration date. It's possible to run the task again, by calling
creating a new pending run. However, this strongly discouraged for simplicity.
If a worker processing a task decides to shutdown, detects an internal error, or
determines that the
task.payload is invalid, it can resolve the run as
exception. Depending on the
reason given for the exception resolution, and
whether or not retries have been exhausted a new pending run may be created.
For example, exceptions with reason
malformed-payload will never be retried.
A task may also enter the exception state, if it is canceled, the deadline is exceeded or the worker disappears and retries have been exhausted.
Whenever, a task is immediately retried, the run will be resolved exception, but the task never enters the exception state, and no message is published about the run that is resolved exception. A message signaling exception will only be published if no automatic retry is initiated.
When a task is resolved, completed, failed, or exception, it will sit
task.expires at which point the task will be deleted by