CometD 2 Oort Cloud

CometD 2 Oort Cloud

Any CometD server can become an Oort comet by configuring an instance of org.cometd.oort.Oort.
The org.cometd.oort.Oort instance is associated to the org.cometd.bayeux.server.BayeuxServer instance, and there can be one Oort instance for each BayeuxServer instance.

Oort comets need to know each other's URL in order to connect together and form a cloud.
From CometD 2.3.0 onwards, an automatic discovery mechanism based on multicast is available; for CometD pre 2.3.0 only a static configuration mechanism exists.

Common Configuration

For both static and automatic discovery there exist a set of parameters can be used to configure the Oort instance.
The following is the list of common configuration parameters shared by both the automatic discovery and static configuration servlets:

Automatic Discovery Configuration

Configuration of the automatic discovery mechanism can be done either via code, or by configuring a org.cometd.oort.OortMulticastConfigServlet in web.xml, for example:

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
         version="2.5">

    <servlet>
        <servlet-name>cometd</servlet-name>
        <servlet-class>org.cometd.server.CometdServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>
    <servlet-mapping>
        <servlet-name>cometd</servlet-name>
        <url-pattern>/cometd/*</url-pattern>
    </servlet-mapping>

    <servlet>
        <servlet-name>oort</servlet-name>
        <servlet-class>org.cometd.oort.OortMulticastConfigServlet</servlet-class>
        <load-on-startup>2</load-on-startup>
        <init-param>
            <param-name>oort.url</param-name>
            <param-value>http://host:port/context/cometd</param-value>
        </init-param>
    </servlet>
</web-app>

Since Oort depends on BayeuxServer, the load-on-startup parameter of the OortMulticastConfigServlet must be greater than the one of the CometdServlet.

The mandatory oort.url init parameter must identify the URL at which this Oort comet can be contacted, and it must be the URL served by the CometdServlet of this node.
This URL will be sent to other Oort comets, so it is important that the host part of the URL does not point to "localhost" but to a resolvable host name or to an IP address, so that other comets in the cluster can contact this comet.
Likewise, the context path part of the URL must be configured properly for this web application.

In addition to the common configuration init parameters, OortMulticastConfigServlet supports the configuration of these additional init parameters:

Each comet that is configured with automatic discovery will emit an advertisement (containing the comet URL) every oort.multicast.advertiseInterval milliseconds on the specified multicast address and port (oort.multicast.groupAddress and oort.multicast.groupPort) with the specified time-to-live (oort.multicast.timeToLive).
Advertisements are emitted until the web application is stopped, and only serve to advertise that a new node has appeared. Oort has a built-in mechanism that takes care of membership organization (see below for details).

When enabling the Oort automatic discovery mechanism, you must be sure that:

• Multicast is enabled in the operative system of your choice
• The network interfaces have multicast enabled
• Multicast traffic routing is properly configured

Linux is normally compiled with multicast support in the most common distributions, and network interfaces can be controlled with the ifconfig command to check if they have multicast enabled.
Multicast routing can be checked with the command route -n, and the output should contain a line similar to:

Destination    Gateway    Genmask      Flags    Metric    Ref    Use    Iface
224.0.0.0      0.0.0.0    240.0.0.0      U         0       0      0     eth0

You may also want to force the JVM to prefer an IPv4 stack by setting the system property -Djava.net.preferIPv4Stack=true to facilitate multicast networking.

Static Discovery Configuration

The static discovery mechanism can be used if multicast is not available on the system where CometD is deployed to, or when using CometD pre 2.3.0.

It is more cumbersome to setup and does not allow dynamic discovery of new nodes; where possible, the automatic discovery mechanism should be used instead.

The static discovery configuration can be done either via code, or by configuring an org.cometd.oort.OortStaticConfigServlet (or org.cometd.oort.OortServlet for CometD pre 2.3.0) in web.xml, for example:

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
         version="2.5">

    <servlet>
        <servlet-name>cometd</servlet-name>
        <servlet-class>org.cometd.server.CometdServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>
    <servlet-mapping>
        <servlet-name>cometd</servlet-name>
        <url-pattern>/cometd/*</url-pattern>
    </servlet-mapping>

    <servlet>
        <servlet-name>oort</servlet-name>
        <servlet-class>org.cometd.oort.OortStaticConfigServlet</servlet-class>
        <load-on-startup>2</load-on-startup>
        <init-param>
            <param-name>oort.url</param-name>
            <param-value>http://host:port/context/cometd</param-value>
        </init-param>
    </servlet>
</web-app>

Like for the automatic discovery, the load-on-startup parameter of the OortStaticConfigServlet must be greater than the one of the CometdServlet.

OortStaticConfigServlet supports the common init parameters listed in the previous section, and the following additional init parameters:

Configured in this way, the Oort comet is ready to be part of the Oort cloud, but it's not part of the could yet, since it does not know the URLs of other comets (and there is no automatic discovery).
In order to make the Oort comet part of the Oort cloud, you can configure the oort.cloud init parameter of the OortStaticConfigServlet with a comma separated list of other Oort comet URLs to connect to:

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
         version="2.5">

    <servlet>
        <servlet-name>cometd</servlet-name>
        <servlet-class>org.cometd.server.CometdServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>
    <servlet-mapping>
        <servlet-name>cometd</servlet-name>
        <url-pattern>/cometd/*</url-pattern>
    </servlet-mapping>

    <servlet>
        <servlet-name>oort</servlet-name>
        <servlet-class>org.cometd.oort.OortStaticConfigServlet</servlet-class>
        <load-on-startup>2</load-on-startup>
        <init-param>
            <param-name>oort.url</param-name>
            <param-value>http://host1:port/context/cometd</param-value>
        </init-param>
        <init-param>
            <param-name>oort.cloud</param-name>
            <param-value>http://host2:port/context/cometd,http://host3:port/context/cometd</param-value>
        </init-param>
    </servlet>
</web-app>

Alternatively, it's possible to write custom initialization code (see the section on service integration for suggestions on how to do it) that links the node to the Oort cloud (this may be useful if Oort comet URLs cannot be know a priori, but may be known at runtime), for example:

public class OortConfigurationServlet extends GenericServlet
{
    public void init() throws ServletException
    {
        // Grab the Oort object
        Oort oort = (Oort)getServletContext().getAttribute(Oort.OORT_ATTRIBUTE);

        // Figure out the URLs to connect to, using other discovery means
        List<String> urls = ...;

        // Connect to the other Oort comets
        for (String url : urls)
        {
            OortComet oortComet = oort.observeComet(url);
            if (!oortComet.waitFor(1000, BayeuxClient.State.CONNECTED))
                throw new ServletException("Cannot connect to Oort comet " + url);
        }
    }

    public void service(ServletRequest request, ServletResponse response) throws ServletException, IOException
    {
        throw new ServletException();
    }
}

The OortComet instance returned by Oort.observeComet(url) is a specialized version of BayeuxClient.

Membership Organization

When a Oort comet is instructed to connect to another Oort comet, a bidirectional communication is established.
If cometA is being connected to cometB (for example via oortA.observeComet(urlB)), then an OortComet instance will be created in cometA connected to cometB, and another OortComet instance will be created in cometB connected to cometA.

After this direct bidirectional communication has been established, a special message is broadcasted on the whole Oort cloud (on channel /oort/cloud) where the two comets broadcast their known siblings.
Every node receiving this message that does not know about those siblings will establish a bidirectional communication with them.

For example, imagine that there are two simple Oort clouds, one made of comets A and B and the other made of comets C and D.
When A and C gets connected, they broadcast their siblings (A broadcasts its siblings, now B and C, while C broadcasts its siblings, now A and D). All nodes connected, directly or indirectly, to the broadcaster will receive this message.
When C receives A's siblings it notices that one is itself (so it does nothing since it's already connected to A), but the other is the unknown sibling B, and establishes a bidirectional connection with B as well. Likewise, A receives the sibling broadcast message from C, and connects to D. Each new bidirectional connection that is established triggers a sibling broadcast message on the whole cloud, until all comets are connected to all comets.

If a comet crashes, for example D, then all other comets will detect that and disconnect from the faulty comet.

In this way, an Oort cloud is aware of its members, but it does not do anything useful for the application.
The next section will cover broadcast messages forwarding over the entire cloud.

Authentication

When a Oort comet connects to another Oort comet, it sends a handshake message containing an extension field that is peculiar to Oort, with the following format:

{
    "channel": "/meta/handshake",
    ... /* other usual handshake fields */
    "ext": {
        "org.cometd.oort": {
            "oortURL": "http://halley.cometd.org:8080/cometd",
            "cometURL": "http://halebopp.cometd.org:8080/cometd",
            "oortSecret": "cstw27r+l+XqE62IrNZdCDiUObA="
        }
    }
}

The oortURL field is the URL of the comet that initiates the handshakes; the cometURL field is the URL of the comet that receives the handshake; the oortSecret is the base64 encoding of the SHA-1 digested bytes of the pre-shared secret of the initiating Oort comet (see the section on common configuration above).

These extension fields provide a way for a Oort comet to distinguish a handshake of a remote client (which may be subject to authentication checks) from a handshake performed by remote comet.
For example, let's assume that remote clients always send an extension field containing an authentication token; then it is possible to write an implementation of SecurityPolicy as follows (see also the section about authentication):

public class OortSecurityPolicy extends DefaultSecurityPolicy
{
    private final Oort oort;

    private OortSecurityPolicy(Oort oort)
    {
        this.oort = oort;
    }

    @Override
    public boolean canHandshake(BayeuxServer server, ServerSession session, ServerMessage message)
    {
        // Local sessions can always handshake
        if (session.isLocalSession())
            return true;

        // Remote Oort comets are allowed to handshake
        if (oort.isOortHandshake(message))
            return true;

        // Remote clients must have a valid token
        Map ext = message.getExt();
        return ext != null && isValid(ext.get("token"));
    }
}

The Oort.isOortHandshake(Message) method validates the handshake message and returns true if it is a handshake from another Oort comet that has been configured with the same pre-shared secret. The pre-shared secret must be explicitly set because it defaults to a random string that is different for each Oort comet.

Broadcast Messages Forwarding

Broadcast messages (that is, messages sent to non-meta and non-service channels, see here for further details) are by definition messages that should be received by all clients that subscribed to the channel the message is being sent.

In an Oort cloud, you may have clients connected to different comets but subscribed to the same channel.
If we have clientA connected to cometA, clientB connected to cometB and clientC connected to cometC, then when clientA broadcasts a message we want clientB and clientC to receive that message, and therefore the Oort cloud must forward the message (sent by clientA and received by cometA) to cometB and cometC.

This is accomplished by configuring the Oort configuration servlets to set the oort.channels init parameter to a comma separated list of channels whose messages will be forwarded to the Oort cloud:

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
         version="2.5">

    <servlet>
        <servlet-name>cometd</servlet-name>
        <servlet-class>org.cometd.server.CometdServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>
    <servlet-mapping>
        <servlet-name>cometd</servlet-name>
        <url-pattern>/cometd/*</url-pattern>
    </servlet-mapping>

    <servlet>
        <servlet-name>oort</servlet-name>
        <servlet-class>org.cometd.oort.OortMulticastConfigServlet</servlet-class>
        <load-on-startup>2</load-on-startup>
        <init-param>
            <param-name>oort.url</param-name>
            <param-value>http://host1:port/context/cometd</param-value>
        </init-param>
        <init-param>
            <param-name>oort.channels</param-name>
            <param-value>/stock/**,/forex/*,/alerts</param-value>
        </init-param>
    </servlet>
</web-app>

Alternatively, it is possible to use Oort.observeChannel(String channelName) to instruct a comet to listen for messages on that channel published to one of the known comets it is connected to.

When cometA observes a channel, it means that messages sent on that channel, but received by other comets, are automatically forwarded to cometA.

Note
Message forwarding is not bidirectional; if cometA forwards messages to cometB it is not automatic that cometB forwards messages to cometA.
However, in most cases the Oort comets are configured in the same way by the same initialization code, and therefore all comets will forward the same channels.

With the ability of observing messages published to broadcast channels, an Oort cloud can already implement a simple chat application among users connected to different nodes.
In the example below, when clientA publishes a message on channel /chat (green arrow), it arrives on cometA; since cometB and cometC have been configured to observe channel /chat, they will both receive the message from cometA (green arrows), and therefore they can deliver the chat message to clientB and clientC respectively (green arrows).

If your application only needs to broadcast messages to clients connected to other comets, an Oort instance is all you need.

If you need to send messages directly to particular clients (for example, clientA wants to send a message to clientC but not to clientB, then you need to setup an additional component of the Oort clustering called Seti.