The Cloud42 Notification Mechanism

Introduction & Architecture

Cloud42 offers a push based notification architecture allowing interested event sinks to be notified as soon as an event on an AMI instance occurs. Event messages on the AMI instance can be easily triggered by shell scripts.
A publish/subscribe pattern is used in combination with a topic mechanism, allowing to subscribe to notification messages triggered by certain events.
In the default implementation, the subscribing event sinks are HTTP endpoints that receive their notifications as SOAP messages. However, the architecture of Cloud42 allows to enhance this very easily, so that notifications can be send in various formats using various transportation protocolls.

General Architecture

The image below shows the actors and the general architecture of the Cloud42 notification mechanism.

Cloud42 Management Framework for EC2

Since one of the requirements is supporting different message formats and different transport protocols when sending notification messages to the sinks, there must be a central point that coordinates subscribing endpoints with their properties and transforms these messages into the desired formats. Therefore, sending messages directly from the AMI instances to the interested endpoints is not an option.

Instead, an approach was chosen featuring a "Notification Core" within Cloud42. This way all application logic is concentrated at one single place and kept away from the EC2 instances, so that they can be run fully independently and without any configuration. The core of the notification mechanism consists of three parts that can be separated logically.

The Cloud42 Web service interface provides a method that allows a subscriber to register for notification messages on a certain topic. In this subscription request, an arbitrary endpoint kann be passed as data sink that finally receives the notifications.

The subscription request is processed by the SubscriptionManager part of the core, which is responsible for handling subscription and unsubscription requests as well as for storing the subscribed endpoint references.

Second, an endpoint is offered that receives and processes event messages coming from an EC2 server instance. The address of this endpoint can be configured using the Cloud42 Web service interface. Event messages from AMI instances always have to be sent to the endpoint address specified this way.

Whenever a message arrives at this endpoint within Cloud42, the third part, the NotificationManager is invoked. Its job is to go through the list of subscribed endpoints for the message's topic and to package the message data into the desired format for each particular endpoint. Furthermore, it finally sends the notification messages to the sinks using the respective transportation protocol.

Subscribe & Unsubscribe

Subscription requests as well as unsubscription requests are SOAP messages that are processed by a SubscriptionProcessor. The SubscriptionProcessor is responsible for parsing the SOAP message and gathering the required information, depending on the kind of subscribing endpoint.
A subscription message itself consists of two parts, a fixed part telling the system which SubscriptionProcessor to use and to which topic to subscribe. The variable part contains an endpoint reference in a format that is understandable for the respective SubscriptionProcessor.

Here is an example for such a subscription message by using the default SubscriptionProcessor of Cloud42.

<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:web="http://webservice.cloud42.jw.de">
  <soap:Header/>
  <soap:Body>
    <web:subscribe>
      <web:subscriptionProcessor>
         de.jw.cloud42.core.eventing.subscriptionProcessor.SOAPSubscriptionProcessor
      </web:subscriptionProcessor>
      <web:topic>myTopic</web:topic>
      <!-- web:subscriptionMessage contains the variable part -->
      <web:subscriptionMessage>
        <wse:Subscribe xmlns:wse="http://schemas.xmlsoap.org/ws/2004/08/eventing">
          <wse:Delivery>
            <wse:NotifyTo>
              <wsa:Address xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing">
                http://localhost:8085/monitor
              </wsa:Address>
            </wse:NotifyTo>
          </wse:Delivery>
        </wse:Subscribe>
      </web:subscriptionMessage>
    </web:subscribe>
  </soap:Body>
</soap:Envelope>

The first two child elements of the <web:subscribe> element contain the information which actual SubscriptionProcessor should handle the request and to which topic the subscription belongs.

The second part, provided within <web:subscriptionMessage>, is the specific part that is parsed by the SubscriptionProcessor. Out of the box, Cloud42 comes with a SubscriptionProcesser that is intended to be used for subscribing endpoints that receive their notifications as SOAP messages over HTTP. In this case, the message consists of a subscription request according to the WS-Eventing specification.

Having parsed the subscription message, the SubscriptionProcessor creates a Subscription object accordingly (in our example, it would be a SOAPSubscription) and stores it in the subscriber storage, linked to its topic. The subscriber store may be implemented in different ways, however in the current implementation it uses a database to persist the Subscriptions.

The Subscription entity holds information such as the endpoint reference for notification messages. Furthermore, it contains the logic to transform messages from the internal message format into the desired format and to send it to its endpoint. Getting back to our example, it would wrap the message into a SOAP envelope and send it to the designated endpoint http://localhost:8085/monitor using HTTP.

As a response to the subscription request, the subscriber retrieves a SOAP message containing a subscription id that must be used to identify the subscription when unsubscribing it. This subscription id is unique for each endpoint on each topic, meaning that the same endpoint can subscribe to various topis, but always is identified by another id. The following listing illustrates an example response to a subscription request.

<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
  <soapenv:Body>
     <ns:subscribeResponse xmlns:ns="http://webservice.cloud42.jw.de">
        <ns:return>uuid:99733DEDFB3443D5281222782013924</ns:return>
     </ns:subscribeResponse>
  </soapenv:Body>
</soapenv:Envelope>

The listing below shows the according unsubscribe request.

<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:web="http://webservice.cloud42.jw.de">
  <soap:Header/>
  <soap:Body>
     <web:unsubscribe>
        <web:subscriptionId>uuid:99733DEDFB3443D5281222782013924</web:subscriptionId>
     </web:unsubscribe>
  </soap:Body>
</soap:Envelope>

Since ending a subscription requires nothing more than deleting an entry from the list of subscriptions, there is no need to specify a special SubscriptionProcessor.

For the sake of completness, the response to the unsubscribe request is presented in the next listing.

<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
  <soapenv:Body>
     <ns:unsubscribeResponse xmlns:ns="http://webservice.cloud42.jw.de">
        <ns:return>true</ns:return>
     </ns:unsubscribeResponse>
  </soapenv:Body>
</soapenv:Envelope>

Event Messages

The last aspect to point out is the communication between the AMI instance and the Notification Core.

Event messages from the EC2 server instance must be sent to an endpoint offered by Cloud42 in a predefined XML format using a HTTP POST. The Cloud42 Web service interface offers methods to configure this endpoint: the Cloud42NotificationService contains a method called setEndpointAddressto do this.

For instance, shell scripts can be used on the instance to trigger such messages. The command line tool Curl, available for all Linux based systems, can be invoked very easily. The Amazon Developer Guide illustrates how instance metadata such as the instance id can be accessed for usage within the messages. This way, absolutely no configuration on the EC2 server instance is necessary. The only thing that must be known is the address of the Cloud42 endpoint where to send the event messages.

An event message could look like this:

<message xmlns="http://cloud42.jw.de/message">
  <topic>myTopic</topic>
  <instanceId>i-38e24051</instanceId>
  <timestamp>2008-09-30 11:19:10 GMT</timestamp>
  <text>message text</text>
  <info>additional message info</info>
</message>

The first thing to mention is that messages are transferred using the Content-Type text/xml to identify their payload as XML data.

The message itself simply consists of a <message> element having five child elements carrying the actual event information, including the topic. The topic field is required, all the other fields can be used arbitrary depending on a user's need. For instance, the <info> element could contain a code or an identifier that is picked up by the final receiver of the event message (a subscribed endpoint) for further processing. All values are simple Strings for full flexibility.

The XML schema is presented here:

<?xml version="1.0" standalone="yes"?>
<xs:schema targetNamespace="http://cloud42.jw.de/message" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="message">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="topic" type="xs:string" minOccurs="1"/>
        <xs:element name="instanceId" type="xs:string" minOccurs="0"/>
        <xs:element name="timestamp" type="xs:string" minOccurs="0"/>
        <xs:element name="text" type="xs:string" minOccurs="0"/>
        <xs:element name="info" type="xs:string" minOccurs="0"/>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

As soon as such an event message arrives at the Endpoint component of the Notification Core, it is parsed and transformed into an internal representation. Now the NotificationManager is called, passing this message. Its job is to retrieve all Subscriptions for the current topic from the subscription storage and to initiate the process of sending the notification messages. As explained above, each particular Subscription knows how to transform the event message from the internal format into a format understandable by its endpoint and how to send it there.

Extensibility - SMS Notification

Introduction

The concept of the SubscriptionProcessor and the specific Subscription objects was elaborated in order to empower protocol specific implementations of the algorithms used to transform and send notification messages. Due to the fact that all protocol specific processings are concentrated at one place, it is very easy to hook in user-defined SubscriptionProcessors for arbitrary message formats. By specifying the desired SubscriptionProcessor in the subscription request on a per subscription basis, it is possible to have different kinds of endpoints subscribed at the same time.

This way, extensibility is guaranteed and the Cloud42 notification mechanism becomes adjustable for every purpose.

This section illustrates how to implement a SubscriptionProcessor supporting SMS notification with few lines of code.

Sending Notifications via SMS

Mainly, empowering the notification mechanism to support a new kind of endpoint consists of three steps.

1. Defining a Subscription Message

The first step is to define a subscription message that is going to be parsed by the new SubscriptionProcessor.
In the case of a SMS notification, the subscribing endpoint is just a telephone number, so a subscription request could look like this:

<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:web="http://webservice.cloud42.jw.de">
  <soap:Header/>
  <soap:Body>
    <web:subscribe>
       <web:subscriptionProcessor>
         my.company.SMSSubscriptionProcessor
       </web:subscriptionProcessor>
       <web:topic>myTopic</web:topic>
       <web:subscriptionMessage>
          <tb:Entry xmlns:tb="http://my.company.com/telephonebook">
             <tb:Number>
               408 726 4390
             </tb:Number>
          </tb:Entry>
       </web:subscriptionMessage>
    </web:subscribe>
  </soap:Body>
</soap:Envelope>

As <web:subscriptionProcessor> we have to specify the class that will finally process the subscription request message. In this case, it is the my.company.SMSSubscriptionProcessor that will be implemented in the next step.

The variable part of the message included in the <web:subscriptionMessage> element contains a telephone number representing the subscribing endpoint.

2. Implementing a SubscriptionProcessor

Now we have to implement the SMSSubscriptionProcessor class. It will parse the subscription message and create a corresponding Subscribtion object, namely a SMSSubscription (see next section).

Cloud42 provides a generic SubscriptionProcessor called GenericSubscriptionProcessor that all other SubscriptionProcessors should inherit from, since it offers some generally usable methods, mainly for processing unsubscription requests and handling subscription ids.

So the my.company.SMSSubscriptionProcessor looks like shown in the listing below.

package my.company;

//imports are missing due to readability

public class SMSSubscriptionProcessor extends GenericSubscriptionProcessor {

    /**
     * This method parses the subscriptionMessage element, extracts the subscribing endpoint (a telephone number) and returns a SMSSubscription object.
     *
     */

    @Override
    public Subscription getSubscriberFromMessage(OMElement message) throws EventingException{
   
        //create SMSSubscription instance
        SMSSubscription subscription = new SMSSubscription();
       
        //parse message, extract telephone number
        OMElement entryElement = message.getFirstChildWithName(new QName(
                Constants.TELEPHONEBOOK_NAMESPACE,
                Constants.ElementNames.Entry));
        if (entryElement == null)
            throw new EventingException("Entry element is not present");

        OMElement numberElement = entryElement
                .getFirstChildWithName(new QName(
                        Constants.TELEPHONEBOOK_NAMESPACE,
                        Constants.ElementNames.Number));
        if (numberElement == null)
            throw new EventingException("Number element is not present");

        String number = numberElement.getText();


        //set the telephone number 
        subscription.setTelephoneNumber(number);
       
       
        return subscription;
   
    }
   
}

3. Implementing the Subscription Class

As a last step, we have to implement the SMSSubscription class containing the algorithms to convert and send the notification messages as SMS. For this purpose, Cloud42 offers an abstract superclass called Subscription that determines the methods a Subscription class must implement. Furthermore, it provides methods for getting and setting both the subscription Id and the topic. The following listing shows how an actual implementation could look like.

package my.company;
//imports are missing due to readability
@Entity //mark as entity in order to be able to save SMSSubscriptons in the datatbase
public class SMSSubscription extends Subscription {
    /**
     * Telephone number where notifications must be sent to.
     */

    private String number;
    /**
     * @return the telephone number for this subscription
     */

    public String getTelephoneNumber() {
        return number;
    }
    /**
     * @param number the telephone number to set for this subscription
     */

    public void setTelephoneNumber(String number) {
        this.number = number;
    }
    /**
     * Transforms the provided notification message into a SMS and sends it.
     *
     * @param message internal representation of the message to send.
     */

    @Override
    public void sendEventData(Message message) throws Exception {
        //send a SMS to the subscribed telephone number
       
        //create human readable SMS text with message information
        String text = "Notification message on topic " + message.topic
                + " from AMI instance " + message.instanceId
                + ". Time: " + message.timestamp + ". Event description is: "
                + message.text + ". Additional information: " + message.info;
           
        //now send this SMS using a fictive SMS library
        MySMSLibrary.sendSMS(number, text)
    }
}

Finally, Hibernate requires a mapping for the newly created class in order to be able to persist it in the database. Thanks to using Hibernate Annotations, this can be done by simply adding the line presented in below to the file hibernate.cfg.xml that is contained in module-hibernate.

<mapping class="my.company.SMSSubscription"/>

That is all that has to be done. There is no need to change any existing code or to adjust any further configuration file. By sending request messages like the one in the first section, a subscriber now can register any mobile phone number for retrieving notifications on the specified topic via SMS.