What is Taskcluster?
Taskcluster is a set of components that manages task queuing, scheduling, execution and provisioning of resources. It was designed to run automated builds and tests at Mozilla.
Like most software organizations, Mozilla needs to compile and test the code its developers produce. The scale of this work is staggering: many days of computer time are required for every change, and developers make changes many times per day, every day. Taskcluster supports execution of all of those build and test jobs quickly (so developers get the information they need) and efficiently (so we do not waste our precious resources). To support the rapid pace of change, Taskcluster is self-serve: you can accomplish just about anything you need to yourself, when you need it. And the service is a Mozilla product: open and transparent, secure and reliable.
The "Taskcluster platform" supports the execution of tasks: the fundamental purpose of Taskcluster. But it is the "core services" which make Taskcluster useful. These provide the tools and integrations necessary to support our complex build, test, and release processes. These include
- Integrations with version-control systems like Mercurial and Git, to start tasks when developers commit code
- Indexes of completed tasks, to find the results of build and test runs
- Hooks to schedule tasks at specific times (e.g., a nightly build)
- Secure storage for secret data such as API keys required to complete tasks
- Workers of various types that will actually run the tasks.
Is it Mozilla-specific?
In principle, no, Taskcluster could be useful to any organization with complex CI problems to solve. While we have built some Mozilla-specific integrations, the platform and many of the core services could be re-used without change in another organization.
We are working hard to make it easy to create a Taskcluster deployment for your own organization. This poses some interesting technical and operational challenges!