« Back to FrontPage

SHIWA concepts

CGI - coarse-grained interoperability
Nesting of different workflow systems in order to achieve execution frameworks interoperability.

 

The coarse-grained approach treats workflow engines as distributed black box systems, where complete sub-workflows are sent to pre-existing enactment engines.

The SHIWA project has developed a workflow embedding and invocation solution that enables users to create P-GRADE native (or parent) workflows that can incorporate non-native (or child) workflows of other workflow systems.
The solution is based on the integration of the P-GRADE portal and workflow system and the GEMLCA application repository and submitter to deploy and invoke different workflow engines. Although the solution is based on the P-GRADE workflow system, GEMLCA can be integrated with other workflow systems. Therefore, it can be considered as a general solution that can be adopted by other workflow systems through integration of the GEMLCA client to them.
This integration will enable other workflow systems to execute non-native workflows if their workflow engines are deployed in a GEMLCA Service. Based on previous work, Kepler, Taverna, and Triana engines were already connected to P-GRADE via GEMLCA.

The CGI approach supports two execution modes. In workflow engine invocation mode the workflow engine is pre-deployed on a dedicated machine. In the workflow engine submission mode workflow engines are submitted to computational resources upon invocation. The SHIWA project will support both modes.

FGI - fine-grained interoperability
Transformation workflow representations in order to achieve workflows migration from one system to another.

 

The fine-grained approach addresses language interoperability by defining an intermediate representation to be used for translation of workflows across systems such as ASKALON, Pegasus, P-GRADE, MOTEUR, Triana.

Fine-grained interoperability focuses on the transformation of workflow representations in order to achieve workflows migration from one system to another. The power of the fine-grained workflow interoperability stands in exploiting the most appropriate enactor for a certain workflow application, independently from the language in which it was created.
 
Currently, each workflow system comes with its own input language designed to satisfy the needs of its specific target community. Workflows are specified in different systems at various levels of detail, sometimes hiding the underlying infrastructure, and sometimes exposing at least part of the system.
In most cases, however, workflows are hard-coded to the workflow system within which they have been developed. Existing language specifications range from simple and pragmatic scripting languages (e.g. shell, Python), to custom DAG-based representations (e.g. DagMan), or more modern XML-based descriptions (e.g. AGWL, GWENDIA, P-GRADE, SCUFL, Triana). The control flow-based abstractions range from pure DAG specifications to more comprehensive imperative constructs such as conditional or loop statements (sequential and parallel). Other approaches based on data flow specifications include advanced collection or array-based data distributions and computations, or even data streaming constructs.
 
It is widely believed that imposing a single standard for the specification of scientific workflows is a difficult task that is likely not to succeed in being adopted by all communities given the heterogeneous nature of their fields and problems to solve. However, an agreement on an Interoperable Workflow Intermediate Representation (IWIR) sufficient for describing all workflows at a lower level of abstraction that is only processed by workflow systems could be more successful as it is not directly exposed to a human developer.
 
More information about IWIR can be found here.
0 Attachments
3941 Views
Average (0 Votes)
Comments

SHIWA (project number 261585) is supported by a Grant from the European Commission's  FP7 INFRASTRUCTURES-2010-2 call under grant agreement n°261585
Provided by the MTA SZTAKI Computer  and Automation Research Institute - Maintained by Laboratory of Parallel and Distributed Computing with Free and Open Source Software (2010)