Notification Service in LEAD
Introduction
WS-Messenger is used in LEAD project to allow remote applications to send notification messages to each other. The notification message format follows WS-eventing specification (8/2004). WS-Eventing is chosen since it is simpler than WS-Notification and it satisfies our needs.
For general document about setting up WS-Messenger and its usage,
please see the "User Guide" web page.
The status of notification service in LEAD is avaiable at the
LEAD
SOA status page. Event model of LEAD project is available at here.
Compatibility with WS-Eventing specification
WS-Messenger
has implemented the basic functions of WS-Eventing specification. The
implemented functions that are currently used include:
The message format of these operations follows WSE specification. This document(sample
messages on the wire) shows the messages on the wire for Subscribe,
Unsubscribe and Notification message. They are the SOAP messages
exchanged in the test sample application (run_TestController.sh) included in the WS-Messenger distribution. The message format follows WSE specification.
Sample messages exchanged in LEAD project is available here.
However, some operations defined in WSE specification are not currently implemented. These operations are:
-
Three operations related to lifetime management of subscription:
Renew, GetStatus and Subscription End. Currently, the lifetime of the subscription is managed manually using a
broker administrator web page. Subscription can be deleted either by unsubscribe operation defined in WSE specification or use the
web page to select the subscriptions using check boxes and delete them together.
Note: The subscription response message does contain expire time of the
subscription according to the WSE specification. However, it is not
actually enforced in the broker. It is only used to follow WSE
specification.
-
Faults messages. Eight
fault messages are defined in the WSE specification. Since we are
currently using WS-Messenger in a controlled environment, they are not
implemented. Java exception may be thrown in the server in these fault
situations.
-
Implied filter (XPath filter) is not implemented. Currently topic based filter is used.
Filter used in subscription message
Topic-based
subscription is currently used LEAD application. The available topic
wildcards depend on underlying messaging system. ActiveMQ is the current
underlying messaging system. It supports several wildcards as described here.
The dialect is the simpleExpression of WS-Topic defined in WS-Notification specification. The dialect is:
'http://www.ibm.com/xmlns/stdwip/web-services/WS-Topics/TopicExpression/simple'.
Event schema used in LEAD
The LEAD event schema currently defines different notification types that are generated
during the execution of the workflow. These workflow notifications are used to track the
progress of the workflow, to track the provenance of the data products generated from
the workflow, and to gather computation time and network latency statistics.
The schema file is available at: LEAD Notification Types XSD. Some sample messages using this schema is available in this document. Additional types will be
defined in future as the need arises, for e.g. by data mining applications indicate a weather
event.
In the section below, workflow engine refers to a BPEL service or Jython script that
initiates and controls the workflow. The workflow itself may be represented in BPEL
or compiled into a Jython script. Service refers to a web service that usually acts
as a wrapper for the user's application code. Application script is the Jython script
that stages files and sets up the execution environment, and consumes and produces
files on behalf of the user application. User application is the executable
that does the real scientific work.
Some of the common fields in the notifications are:
- topic: The notification topic to which this notification being published to.
- WorkflowID: This is a unique string that identifies the workflow instance that is
generating this notification. It is currently set to the notification topic for that workflow
instance.
- NodeID: This is a string that uniquely identifies the node within the workflow that
is generating this notification. This ID is assigned statically when the workflow is composed.
- WorkflowTimestep: The workflow engine maintains a logical clock that monotonically
increases with every service invocation by the workflow engine. The workflow timestep field is
the timestep of this clock for the current service's invocation. This allows for temporal ordering
of the notifications in the absence of a globally synchronized clock..
- ServiceID: The unique QName of the concrete WSDL representing the service instance
that is generating this notification.
- timestamp: The timestamp on the client at the moment this notification is generated.
- fileUUID: the LEAD GUID for a logical file that participates in the workflow. The
physical file that corresponds to this GUID can be resolved using the LEAD Name Resolver. The
actual physical file that caused this notification may be included as a separate field.
The notification types are ennumerated and described in brief below.
- WorkflowStart{topic, workflowID, timestamp}
- Generated by the workflow engine after the workflow is started and before the first
service in the workflow is called.
- WorkflowFinishSuccess{topic, workflowID, timestamp}
- Generated by the workflow engine after all services in the workflow finish successfully and
hence the workflow itself completes successfully.
- WorkflowFinishFailed{topic, workflowID, timestamp,
errorMessage, stackTrace}
- Generated by the workflow engine if any of the services in the workflow fail or if the workflow
fails for some other reason and terminates.
'errorMessage' is a human readable description of the error while 'stackTrace' is the stack trace
optionally dumped by the faulty region.
- InvokeServiceStart{topic, workflowID, nodeID, wfTimeStep, serviceID, timestamp}
- Generated by the workflow engine before it invokes a service.
- InvokeServiceSuccess{topic, workflowID, nodeID, wfTimeStep, serviceID, timestamp}
- Generated by the workflow engine after the invovation to a service returns successfully.
- InvokeServiceFailed{topic, workflowID, nodeID, wfTimeStep, serviceID, timestamp,
errorMessage, stackTrace}
- Generated by the workflow engine if the invocation of a service failed.
'errorMessage' is a human readable description of the error while 'stackTrace' is the stack trace
optionally dumped by the fault.
- ApplicationStart{topic, workflowID, nodeID, wfTimeStep, serviceID, timestamp,
name, host}
- Generated by the application script when it is invoked by the service and starts executing.
'name' is a human readable name for the application and 'host' is the hostname for the machine on
which the application is executing.
- ApplicationFinishSuccess{topic, workflowID, nodeID, wfTimeStep, serviceID, timestamp,
applicationResult}
- Generated by the application script when it finishes executing successfully.
'applicationResult' is a list of name-value pairs that are the results of this application, that may
be used by successive services as inputs.
- ApplicationFinishFailed{topic, workflowID, nodeID, wfTimeStep, serviceID, timestamp,
errorMessage, stackTrace}
- Generated by the application script when it terminates executing due to an exception.
'errorMessage' is a human readable description of the error while 'stackTrace' is the stack trace
optionally dumped by the faulty section.
- ComputationDuration{topic, workflowID, nodeID, wfTimeStep, serviceID, timestamp,
name, durationMillis}
- Generated by the application script when it completes executing a particular computation section,
possibly the user's application, and the time taken for that section is available.
'name' is a human readable description of the computation section while 'durationMillis' is the
time in milliseconds taken by that section.
- FileSendDuration & FileReceiveDuration{topic, workflowID, nodeID, wfTimeStep, serviceID, timestamp, fileUUID,
localFilePath, fileSizeBytes, remoteFilePath, durationMillis}
- Generated by the application script when it completes sending (receiving) a local file to (from) a remote location,
and the time taken for that transfer is available. 'localFilePath' is the absolute path to the local file that
is sent (received), 'fileSizeBytes' is the physical size of the transfered file in bytes,
'remoteFilePath' is the remote URL to (from) which the file is sent (received), and 'durationMillis' is the
time in milliseconds taken for the transfer.
- FileConsumed{topic, workflowID, nodeID, wfTimeStep, serviceID, timestamp, fileUUID,
localFilePath, fileSizeBytes}
- Generated by the application script when it uses/consumes a local file for its processing.
'localFilePath' is the absolute path to the local file, and 'fileSizeBytes' is the
physical size of the local file.
- FileProduced{topic, workflowID, nodeID, wfTimeStep, serviceID, timestamp, fileUUID,
localFilePath, fileSizeBytes}
- Generated by the application script when it creates/produces a local file during its execution.
'localFilePath' is the absolute path to the local file generated, and 'fileSizeBytes' is the
physical size of the local file.
- PublishURL{topic, workflowID, nodeID, wfTimeStep, serviceID, timestamp,
title, location}
- Generated by the application script when a file that can be visualized or of special interest
is created. This notification usually prompts the notification listener to display the URL as a hyperlink.
'title' is a human readable name or description of the file that can be used as the text for the hyperlink,
and 'location' is a URL to which the hyperlink should point.
- Log{topic, workflowID, nodeID, wfTimeStep, serviceID, timestamp,
message, level}
- Generated by the application script when a generic event of interest occurs. This can be used as a
general purpose tool for logging that is human understandable.
'message' is the message that is being logged, and 'level' is the granularity or importance of this
log message. The levels availabe are INFO, EXCEPTION, WARNING, DEBUG, and these can act as fields on which
the log message is filtered by the notification listener.
Programming Guide for using wsmg package in LEAD application
Detailed programming document for using WS-Messenger in LEAD project is available here (Using WS-Messenger for LEAD project). Packages designed for LEAD project are described in this document.
Administration of Notification Broker at rainier in Extreme Lab
Work in Progress
Currently, we are improving the reliability and scalability of WS-Messenger.
Created: 09/21/05 by Yi Huang
updated: 10/11/2005