This proposal has been approved and the Eclipse TEA™ (Tasking Engine Advanced) project has been created.
Visit the project page for the latest information and development.

Eclipse TEA (Tasking Engine Advanced)

Thursday, April 14, 2016 - 07:19 by Markus Duft
This proposal is in the Project Proposal Phase (as defined in the Eclipse Development Process) and is written to declare its intent and scope. We solicit additional participation and input from the community. Please login and add your feedback in the comments section.
Parent Project
Proposal State
Created
Background

SSI Schaefer's large business applications are complex pieces of software, composed of many plug-ins and many applications. Initially, the PDE build was used (AutoBuild in the IDE, ANT headlessly). This had quite a few problems:

  • Individual/duplicate configuration, a lot of maintenance of property files, …

  • The Autobuild in the IDE itself takes into account “Import-Package” which seems to cause some performance issues.

  • Integration of code generators that are actual applications in the workspace is not trivial/possible with PDE.

  • Actual building of applications requires a lot of manual configuration (creation of features, puzzling together which Plug-Ins belong to which feature/product, versions, etc.)

Although TEA (previously named “WAMAS Power Build” (WpoB)) has Tasks in its core library to solve all these issues, and to facilitate creating reproducible and fast builds for both IDE and headless, TEA evolved over time to be a broader used tasking engine at SSI Schaefer. Nearly all tasks that need to be executed both by developers and headless builds are reflected as tasks. These tasks can be chained together to so-called TaskChains. These can be executed either via a Menu in the IDE, or via a command line argument from a headless RCP application.

The name TEA was chosen despite the fact that most of the already existing tasks deal with build related topics, as Stefan Xenos (Google) expressed some interest in possibly re-integrating some of the build related things back to PDE over time. This leaves the strong tasking model left over.

Scope

The scope of Eclipse TEA is to provide the APIs for the core concepts of tasks and task chains, as well as the execution environment(s) for those. There is an integration into the Eclipse IDE itself (a menu), and a headless application that allow to run the exact same code from IDE or headless. Eclipse TEA provides a core library of predefined tasks and task chains that allow to execute various recurring tasks out of the box (mainly in the area of building software, update sites, ...).

Although TEA currently focuses on providing tasks for building and managing Eclipse Plug-in projects, there is no technical restriction in supporting more project types in the future.

Description

Eclipse TEA is a tasking orchestration engine that can be run from within the Eclipse IDE or headlessly. It is immediately concerned with tasks related to building and managing plug-ins for the Eclipse Platform, but is general enough to support other types of tasks and builds.

Eclipse TEA's major component is the so called TaskingEngine. This engine can be created and run both from the IDE, as well as headlessly (with a headless presentation engine – thus the PlatformUI.getWorkbench() DOES work – in case a task requires “UI” but still should run headlessly). This allows to create reproducible and stable TaskChains that can be used by both developers and automated builds to create the same, reproducible results for certain tasks.

The original core of TEA was building of projects. For this, there is a task to compile all projects in a workspace. This task has the power to:

  • Calculate a build graph based on “Require-Bundle” dependencies. It is ignoring “Package-Import” completely (currently). The background here is, that the IDE autobuild seems to invest a lot of time figuring out dependencies based on “Package-Import”, running in circles to find a proper build order for bundles – even if this is not required. SSI Schaefer uses “Require-Bundle” exclusively, as this is the better option for applications with well-determined environment anyway (better validation possibilities of architecture, …). “Package-Import” is only great for frameworks that “don't know” their final environment that they will be running in – as applications typically compose different frameworks to a single application.

    • Each node in this graph represents a Plugin that is to be built. On execution of the graph, every IProject's .build() method is called to do the very same thing as the IDE autobuild. Only the dependency calculation is done outside – the build is still unmodified standard.

    • TEA allows to inject additional nodes into that graph. These nodes can be contributed via extensions, and are basically free to do whatever they want. Some framework is included that allows to create nodes that wait for certain bundles to be built (for instance a code generator), and server as dependency for other nodes (for instance the bundles where code is generated into). The node itself can then execute a certain program in the workspace. Support for integration with the “Eclipse Launch Configuration Generator” (another SSI Schaefer project proposal) is provided.

Apart from this, TEA has a library of a lot of useful tasks that allow to easily build complex applications. For instance, building a product requires only one time setup with TEA. Create templates for a product feature, create a product, and setup the TEA tasks to build this new product. As a starting point, TEA takes the path to a launch configuration template from the “Eclipse Launch Configuration Generator” project. This launch configuration is examined, dependencies are resolved dynamically, and the feature for the product is automatically generated. Afterwards, the product can be exported with a TEA task, and that's it. Versions are taken care of, etc. There is no need for more configuration. This takes a lot of effort from developers when developing in workspaces with multiple different applications/products. At SSI Schaefer, the business application is composed out of multiple (10s of) RCP products that are managed this way.

Along with the core infrastructure, all Tasks in the core library that are not SSI Schaefer specific will be contributed. This includes but is not limited to:

  • Build graph calculation and execution (building a workspace)

  • Generation of features based on launch configurations (see other Project Proposal “Launch Configuration Generator” - this part depends on it).

  • Generation and export of P2 update sites (also based on above generated features).

  • Generation and export of Eclipse RCP Products (.product). Also uses the generated P2 update sites and features.

  • Lightweight Maven (Gradle, ...) repository integration tasks. Used to resolve dependencies and download JAR files to be integrated into a single plugin (quite limited but useful)

  • ...and many more individual tasks...

Why Here?

The Eclipse Foundation seems like the perfect home for this project, as its sole purpose is to augment the Eclipse Platform. It uses a lot of Eclipse PDE internals, and provides a lot of power to Eclipse IDE users. Also, interest has been expressed by platform committers and also users to see this project come open source.

Future Work
  • A Wizard to create TEA-enabled feature and product projects automatically from existing “Eclipse Launch Configuration Generator” templates.

  • Explore integration possibilities with Eclipse EASE. Maybe it's possible to load TEA tasks as a script resource with EASE for TEA.

  • Explore potential to backport some build related features into PDE.

Project Scheduling

The plan is to prepare the initial contribution within a few weeks after the acceptance of the project. Right after the initial contribution is accepted, build and release infrastructure should be prepared withing another one to two weeks. Depending on the time the reviews and project acceptance take, the plan is to finish the whole process by latest September 2017. This is to be able to present the current state at the EclipseCon Europe 2017

Project Leads
Interested Parties

SSI Schaefer

Stefan Xenos (Google)

kichwa coders (Tracy, Jonah)

Joer Buchberger (PRÜFTECHNIK Condition Monitoring GmbH)

Initial Contribution

There is a complete set of code that makes up a fully working version of this project. This code is in production use since many years now, and thus very stable. There has been (and will be) some refactoring to separate out some things that are special needs to SSI Schaefer. The code will also be refactored/rebranded prior to the initial contribution to no longer carry SSI Schaefer related bundle/package names.

The initial contribution will be split up in various bundles that compromise the core tasking runtime, the core task library, and potentially other bundles.

Source Repository Type