Task Index API Documentation


BaseUrl
https://index.taskcluster.net/v1

The task index, typically available at index.taskcluster.net, is responsible for indexing tasks. The service ensures that tasks can be located by recency and/or arbitrary strings. Common use-cases include:

  • Locate tasks by git or mercurial <revision>, or
  • Locate latest task from given <branch>, such as a release.

Index hierarchy, tasks are indexed in a dot (.) separated hierarchy called a namespace. For example a task could be indexed with the index path some-app.<revision>.linux-64.release-build. In this case the following namespaces is created.

  1. some-app,
  2. some-app.<revision>, and,
  3. some-app.<revision>.linux-64

Inside the namespace some-app.<revision> you can find the namespace some-app.<revision>.linux-64 inside which you can find the indexed task some-app.<revision>.linux-64.release-build. This is an example of indexing builds for a given platform and revision.

Task Rank, when a task is indexed, it is assigned a rank (defaults to 0). If another task is already indexed in the same namespace with lower or equal rank, the index for that task will be overwritten. For example consider index path mozilla-central.linux-64.release-build. In this case one might choose to use a UNIX timestamp or mercurial revision number as rank. This way the latest completed linux 64 bit release build is always available at mozilla-central.linux-64.release-build.

Note that this does mean index paths are not immutable: the same path may point to a different task now than it did a moment ago.

Indexed Data, when a task is retrieved from the index the result includes a taskId and an additional user-defined JSON blob that was indexed with the task.

Entry Expiration, all indexed entries must have an expiration date. Typically this defaults to one year, if not specified. If you are indexing tasks to make it easy to find artifacts, consider using the artifact's expiration date.

Valid Characters, all keys in a namespace <key1>.<key2> must be in the form /[a-zA-Z0-9_!~*'()%-]+/. Observe that this is URL-safe and that if you strictly want to put another character you can URL encode it.

Indexing Routes, tasks can be indexed using the API below, but the most common way to index tasks is adding a custom route to task.routes of the form index.<namespace>. In order to add this route to a task you'll need the scope queue:route:index.<namespace>. When a task has this route, it will be indexed when the task is completed successfully. The task will be indexed with rank, data and expires as specified in task.extra.index. See the example below:

{
  payload:  { /* ... */ },
  routes: [
    // index.<namespace> prefixed routes, tasks CC'ed such a route will
    // be indexed under the given <namespace>
    "index.mozilla-central.linux-64.release-build",
    "index.<revision>.linux-64.release-build"
  ],
  extra: {
    // Optional details for indexing service
    index: {
      // Ordering, this taskId will overwrite any thing that has
      // rank <= 4000 (defaults to zero)
      rank:       4000,

      // Specify when the entries expire (Defaults to 1 year)
      expires:          new Date().toJSON(),

      // A little informal data to store along with taskId
      // (less 16 kb when encoded as JSON)
      data: {
        hgRevision:   "...",
        commitMessae: "...",
        whatever...
      }
    },
    // Extra properties for other services...
  }
  // Other task properties...
}

Remark, when indexing tasks using custom routes, it's also possible to listen for messages about these tasks. For example one could bind to route.index.some-app.*.release-build, and pick up all messages about release builds. Hence, it is a good idea to document task index hierarchies, as these make up extension points in their own.

Functions

For more information on invoking the API methods described here, see Using the APIs in the manual.
SignatureSummary
findTask(indexPath) : resultFind Indexed Task
listNamespaces(namespace, payload) : resultList Namespaces
listTasks(namespace, payload) : resultList Tasks
insertTask(namespace, payload) : resultInsert Task into Index
findArtifactFromTask(indexPath, name) : voidGet Artifact From Indexed Task
ping() : voidPing Server

Find Indexed Task

Method
get
Route
/task/<indexPath>
Signature
findTask(indexPath) : result
Stability
stable

Find a task by index path, returning the highest-rank task with that path. If no task exists for the given path, this API end-point will respond with a 404 status.

Response

Indexed Task Response (source)

Representation of an indexed task.

namespacestring[0:255]

Namespace of the indexed task, used to find the indexed task in the index.

taskIdstring^[A-Za-z0-9_-]{8}[Q-T][A-Za-z0-9_-][CGKOSWaeimquy26-][A-Za-z0-9_-]{10}[AQgw]$

Unique task identifier, this is UUID encoded as URL-safe base64 and stripped of = padding.

ranknumber

If multiple tasks are indexed with the same namespace the task with the highest rank will be stored and returned in later requests. If two tasks has the same rank the latest task will be stored.

dataObject of

Representation of an indexed task.

Anything ¯\_(ツ)_/¯
expiresstringdate-time

Date at which this entry expires from the task index.



List Namespaces

Method
post
Route
/namespaces/<namespace>
Signature
listNamespaces(namespace, payload) : result
Stability
stable

List the namespaces immediately under a given namespace.

This endpoint lists up to 1000 namespaces. If more namespaces are present, a continuationToken will be returned, which can be given in the next request. For the initial request, the payload should be an empty JSON object.

Request Payload

List Namespaces Request (source)

Request to list namespaces within a given namespace.

limitinteger[1:1000]
default: 1000

Maximum number of results per page. If there are more results than this a continuation token will be return.

continuationTokenstring

A continuation token previously returned in a response to this list request. This property is optional and should not be provided for first requests.

Response

List Namespaces Response (source)

Response from a request to list namespaces within a given namespace.

namespacesArray of

List of namespaces.

namespacestring[0:255]

Fully qualified name of the namespace, you can use this to list namespaces or tasks under this namespace.

namestring

Name of namespace within it's parent namespace.

expiresstringdate-time

Date at which this entry, and by implication all entries below it, expires from the task index.

continuationTokenstring

A continuation token is returned if there are more results than listed here. You can optionally provide the token in the request payload to load the additional results.



List Tasks

Method
post
Route
/tasks/<namespace>
Signature
listTasks(namespace, payload) : result
Stability
stable

List the tasks immediately under a given namespace.

This endpoint lists up to 1000 tasks. If more tasks are present, a continuationToken will be returned, which can be given in the next request. For the initial request, the payload should be an empty JSON object.

Remark, this end-point is designed for humans browsing for tasks, not services, as that makes little sense.

Request Payload

List Tasks Request (source)

Request to list tasks within a given namespace.

limitinteger[1:1000]
default: 1000

Maximum number of results per page. If there are more results than this a continuation token will be return.

continuationTokenstring

A continuation token previously returned in a response to this list request. This property is optional and should not be provided for first requests.

Response

List Tasks Response (source)

Representation of an indexed task.

tasksArray of

List of tasks.

namespacestring[0:255]

Index path of the task.

taskIdstring^[A-Za-z0-9_-]{8}[Q-T][A-Za-z0-9_-][CGKOSWaeimquy26-][A-Za-z0-9_-]{10}[AQgw]$

Unique task identifier for the task currently indexed at namespace.

ranknumber

If multiple tasks are indexed with the same namespace the task with the highest rank will be stored and returned in later requests. If two tasks has the same rank the latest task will be stored.

dataObject of

Representation of a task.

Anything ¯\_(ツ)_/¯
expiresstringdate-time

Date at which this entry expires from the task index.

continuationTokenstring

A continuation token is returned if there are more results than listed here. You can optionally provide the token in the request payload to load the additional results.



Insert Task into Index

Method
put
Route
/task/<namespace>
Scopes
index:insert-task:<namespace>
Signature
insertTask(namespace, payload) : result
Stability
stable

Insert a task into the index. If the new rank is less than the existing rank at the given index path, the task is not indexed but the response is still 200 OK.

Please see the introduction above for information about indexing successfully completed tasks automatically using custom routes.

Request Payload

Insert Task Request (source)

Representation of an a task to be indexed.

taskIdstring^[A-Za-z0-9_-]{8}[Q-T][A-Za-z0-9_-][CGKOSWaeimquy26-][A-Za-z0-9_-]{10}[AQgw]$

Unique task identifier, this is UUID encoded as URL-safe base64 and stripped of = padding.

ranknumber

If multiple tasks are indexed with the same namespace the task with the highest rank will be stored and returned in later requests. If two tasks has the same rank the latest task will be stored.

dataObject of

Representation of an a task to be indexed.

Anything ¯\_(ツ)_/¯
expiresstringdate-time

Date at which this entry expires from the task index.

Response

Indexed Task Response (source)

Representation of an indexed task.

namespacestring[0:255]

Namespace of the indexed task, used to find the indexed task in the index.

taskIdstring^[A-Za-z0-9_-]{8}[Q-T][A-Za-z0-9_-][CGKOSWaeimquy26-][A-Za-z0-9_-]{10}[AQgw]$

Unique task identifier, this is UUID encoded as URL-safe base64 and stripped of = padding.

ranknumber

If multiple tasks are indexed with the same namespace the task with the highest rank will be stored and returned in later requests. If two tasks has the same rank the latest task will be stored.

dataObject of

Representation of an indexed task.

Anything ¯\_(ツ)_/¯
expiresstringdate-time

Date at which this entry expires from the task index.



Get Artifact From Indexed Task

Method
get
Route
/task/<indexPath>/artifacts/<name>
Scopes
queue:get-artifact:<name>
Signature
findArtifactFromTask(indexPath, name) : void
Stability
stable

Find a task by index path and redirect to the artifact on the most recent run with the given name.

Note that multiple calls to this endpoint may return artifacts from differen tasks if a new task is inserted into the index between calls. Avoid using this method as a stable link to multiple, connected files if the index path does not contain a unique identifier. For example, the following two links may return unrelated files:

This problem be remedied by including the revision in the index path or by bundling both installer and debug symbols into a single artifact.

If no task exists for the given index path, this API end-point responds with 404.



Ping Server

Method
get
Route
/ping
Signature
ping() : void
Stability
stable

Respond without doing anything. This endpoint is used to check that the service is up.