CometD JavaScript

CometD JavaScript Implementation

The JavaScript implementation of the Bayeux specification and API has been totally rewritten starting from CometD version 1.0.beta8, and further refactored since 1.0.beta9.
What is available now is a portable JavaScript implementation with bindings for the major JavaScript toolkits, currently Dojo and jQuery.

What this means is that the CometD Bayeux JavaScript implementation is written in pure JavaScript with no dependencies on the toolkits, and that the toolkit bindings add the syntactic sugar that makes the Bayeux APIs feel like they are native to the toolkit.
For example, it is possible to refer to the standard cometd object using the following notation:

// Dojo style
var cometd = dojox.cometd;

// jQuery style
var cometd = $.cometd;

If you followed the Primer, you may have noticed that the skeleton project now requires to reference both the portable implementation, under org/cometd.js, and one binding - for example Dojo's - under dojox/cometd.js. For jQuery, the binding is under jquery/jquery.cometd.js.

The usage of the Bayeux APIs from the JavaScript toolkits is almost identical, and in the following we will not refer to a particular toolkit.
Small differences only surface when passing callback functions to the Bayeux API, where Dojo users may like to use dojo.hitch(), while jQuery users may like an anonymous function approach.

The following sections will go in detail about the JavaScript Bayeux APIs and their implementation secrets.

CometD JavaScript Configuration

JavaScript CometD API: Configuration and Initialization

After you have setup your skeleton project following the Primer, you may want to fully understand how to customize and configure the parameters that govern the behavior of the Cometd implementation.

The whole API is available through a single object prototype named org.cometd.Cometd.
The Dojo toolkit has one instance of this object available under the name dojox.cometd, while for jQuery it is available under the name $.cometd.

This default cometd object has been instantiated and configured with the default values and it has not started any Bayeux communication yet.
Before it can start any Bayeux communication it needs a mandatory parameter: the URL of the Bayeux server.

There are 2 ways of passing this parameter:

// First style: URL string
cometd.configure('http://localhost:8080/cometd');

// Second style: configuration object
cometd.configure({
    url: 'http://localhost:8080/cometd'
});

The first way is a shorthand for the second way.
However, the second way allows to pass other configuration parameters, currently:

Parameter Name Required Default Value Parameter Description
url yes The URL of the Bayeux server this client will connect to
logLevel no info The log level. Possible values are: "warn", "info", "debug". Output to window.console if available
maxConnections no 2 The max number of connections used to connect to the Bayeux server.
Only change this value if you know exactly what is the client's connection limit and what "request queued behind long poll" means
backoffIncrement no 1000 The number of milliseconds of which the backoff time is incremented every time a connection with the Bayeux server fails.
A reconnection will be attempted after the backoff time elapses
maxBackoff no 60000 The max number of milliseconds of the backoff time after which the backoff time is not incremented anymore
reverseIncomingExtensions no true Controls whether the incoming extensions will be called in reverse order with respect to the registration order
maxNetworkDelay no 10000 The max number of milliseconds to wait before considering a request to the Bayeux server failed.
requestHeaders no {} An object containing the request headers to be sent for every bayeux request (for example: {"My-Custom-Header":"MyValue"})
appendMessageTypeToURL no true Whether or not the Bayeux message type (handshake, connect, disconnect) is appended to URL of the Bayeux server (see above).
autoBatch no false Whether multiple publishes that gets queued up will be sent as a batch on the first occasion, without requiring explicit batching.

 
After you have configured the cometd object, it has not started the Bayeux communication yet. To start the Bayeux communication, you need to call handshake(), see the next section.

Previous users of the JavaScript Cometd implementation were used to call a method called init(). This method still exists, and it is a shorthand for calling configure() followed by handshake().
Follow the advices in the next section as they apply as well to init().

CometD JavaScript Handshake

JavaScript CometD API: Handshake

The call to handshake() (or to init()) is the one that initiates the Bayeux communication with the Bayeux server.

The Bayeux handshake performs 2 tasks:

  • the client and the server negotiate the type of transport to use
  • once the transport is negotiated successfully, the server informs the client with the detailed timings of the requests

As with several methods of the JavaScript CometD API, it is an asynchronous method: it returns immediately, well before the Bayeux handshake steps have completed.

Note
Calling handshake() does not mean that you have completed the handshake with the server when handshake() returns.

The handshake may fail for several reasons:

  • you mistyped the server URL
  • the transport could not be negotiated successfully
  • the server denied the handshake (for example, the authentication credentials were wrong)
  • the server crashed
  • there was a network failure

Therefore it is not a good idea to write this code:

// Configure and handshake
cometd.init('http://localhost:8080/cometd');

// Publish to a channel
cometd.publish('/foo', { foo: 'bar' });

It is not a good idea, because there is no guarantee that the call to publish() (which we cover in a later section) can actually succeed in contacting the Bayeux server.
Since the API is asynchronous, you have no way of knowing synchronously (i.e. by having handshake() return an error code or by throwing an exception) that the handshake failed.
Even if the handshake succeeds, you may still be "disconnected" from the Bayeux server, for example because the server crashed just after the successful handshake.

Fortunately there is a way to be notified about the details of the Bayeux protocol message exchange: by adding listeners to special channels (called meta channels).
This is explained in the section about subscriptions.

CometD JavaScript Subscription

JavaScript CometD API: Subscribing and Unsubscribing

Channels

The Bayeux specification defines the concept of a channel: it is like a messaging topic where interested parties can subscribe to receive information published onto the channel.
There are 3 types of channels:

  • meta channels
  • service channels
  • normal channels

A channel looks like a directory path such as /meta/connect (a meta channel; all meta channels starts with the prefix /meta/), or /service/chat (a service channel; all service channels starts with the prefix /service/) or /foo/bar (a normal channel).

Meta Channels

Meta channels are created by the Bayeux protocol itself.
It is not possible to subscribe to meta channels: the server will reply with an error message. However, it is possible to listen to meta channels (see below the difference between subscribing and listening).
It makes no sense to publish messages to meta channels: only the Bayeux protocol implementation creates and sends messages on meta channels.
Meta channels are useful on the client to listen for error messages like handshake errors (for example because the client did not provide the correct credentials) or network errors (for example to know when the connection with the server has broken or when it has been re-established).

Service Channels

Service channels are used in the case of request/response style of communication between client and server (as opposed to the publish/subscribe style of communication or normal channels).
While subscribing to service channels yields no errors, this is a no-operation for the server: the server ignores the subscription request.
It is possible to publish to service channels, with the semantic of a communication between a specific client (the one that's publishing the message on the service channel) and the server.
Service channels are useful to implement, for example, private chat messages: in a chat with userA, userB and userC, userA can publish a private message to userC (without userB knowing about) using service channels.

Normal Channels

Normal channels have the semantic of a messaging topic and are used in the case of publish/subscribe style of communication.
Usually, it is possible to subscribe to normal channels and to publish to normal channels; this can only be forbidden using a security policy on the Bayeux server.
Normal channels are useful to implement broadcasting of messages to all subscribed clients, for example in case of a stock price change.

Subscribers versus Listeners

The JavaScript CometD API has 2 APIs to work with channel subscriptions:

  • addListener() and the correspondent removeListener()
  • subscribe() and the correspondent unsubscribe()

The addListener() method:

  • must be used to listen to meta channel messages
  • may be used to listen to service channel messages (you may also use subscribe())
  • should not be used to listen normal channel messages (use subscribe() instead)
  • does not involve any communication with the Bayeux server, and as such can be called before calling handshake()
  • is synchronous: when it returns, you are guaranteed that the listener has been added

The subscribe() method:

  • must not be used to listen to meta channels messages (otherwise the server will return an error)
  • may be used to listen to service channel messages (you may also use addListener())
  • should be used to listen to normal channel messages
  • involves a communication with the Bayeux server and as such cannot be called before calling handshake()
  • is asynchronous: it returns immediately, well before the Bayeux server has received the subscription request

Note
Calling subscribe() does not mean that you have completed the subscription with the server when subscribe() returns.

Both addListener() and subscribe() return a subscription object that must be passed to, respectively, removeListener() and unsubscribe():

// Some initialization code
var subscription1 = cometd.addListener('/meta/connect', function() { ... });
var subscription2 = cometd.subscribe('/foo/bar/', function() { ... });

// Some de-initialization code
cometd.unsubscribe(subscription2);
cometd.removeListener(subscription1);

A common pattern of utilization is to handle the subscription code in an idempotent method, like this:

var _subscription;

// The idempotent method
function _refresh()
{
    _appUnsubscribe();
    _appSubscribe();
}

function _appUnsubscribe()
{
    if (_subscription) 
        cometd.unsubscribe(_subscription);
    _subscription = null;
}

function _appSubscribe()
{
    _subscription = cometd.subscribe('/foo/bar', function() { ... });
}

The same of course applies also for addListener()/removeListener().

The point is that you have to be careful in your application: you have to remove your subscriptions, in order to avoid to leak functions, or to execute functions more than once (since you could erroneously bind the same callback twice).
Refer also to the Primer for a discussion about using idempotent methods.

How do subscribe() and unsubscribe() behave in case the Bayeux server is not reachable (due to network failures or because the server crashed) ?
In subscribe() the local listener is first added to the list of subscribers for that channel, then the server communication is attempted. If the communication fails, the server will not know that it has to send messages to this client and therefore on the client the local listener (although present) will never be invoked.
In unsubscribe(), the local listener is first removed from the list of subscribers for that channel, then the server communication is attempted. If the communication fails, the server will still send the message to the client but there will be no local listener to dispatch to.

Listeners/Subscribers Exception Handling

If a listener or subscriber function throws an exception (for example, calls a method on an undefined object, etc.), then the error message is logged (at level "debug").
However, there is a way to intercept these errors by defining the global listener exception handler, that is invoked every time a listener or subscriber throws an exception:

cometd.onListenerException = function(exception, subscriptionHandle, isListener, message)
{
    // Uh-oh, something went wrong, disable this listener/subscriber
    // Object "this" points to the CometD object
    if (isListener)
        this.removeListener(subscriptionHandle);
    else
        this.unsubscribe(subscriptionHandle);
}

It is be possible to send messages to the server from the listener exception handler.
If the listener exception handler itself throws an exception, this exception is logged at level "info" and the CometD implementation will not break.
Note that a similar mechanism exists for extensions, see here.

Wildcard Subscriptions

It is possible to subscribe to several channels at once using wildcards, like this:

cometd.subscribe("/chatrooms/*", function(message) { ... });

A single asterisk has the meaning of matching a single channel segment, so in the example above it will match channels /chatrooms/12 and /chatrooms/15, but not /chatrooms/12/upload.
To match multiple channel segments, use the double asterisk:

cometd.subscribe("/events/**", function(message) { ... });

With the double asterisk, the channels /events/stock/FOO and /events/forex/EUR will match, as well as /events/feed and /events/feed/2009/08/03.

The wildcard mechanism works also for listeners, so it is possible to listen to all meta channels like this:

cometd.addListener("/meta/*", function(message) { ... });

By default, subscriptions to the global wildcards /* and /** result in an error, but this behavior can be changed by specifying a custom security policy on the Bayeux server.

The wildcards can only be specified as last segment of the channel, so these are invalid subscriptions: /**/foo or /foo/*/bar.

Meta Channel List

These are the meta channels available in the JavaScript CometD implementation:

  • /meta/handshake
  • /meta/connect
  • /meta/disconnect
  • /meta/subscribe
  • /meta/unsubscribe
  • /meta/publish
  • /meta/unsuccessful

Each meta channel is notified when the correspondent Bayeux message is handled by the JavaScript Cometd implementation.
The /meta/unsuccessful channel is notified in case of any failure.

By far the most interesting meta channel to subscribe to is /meta/connect, because it gives the status of the current connection with the Bayeux server. In combination with /meta/disconnect, it can be used, for example, to display a green "connected" icon or a red "disconnected" icon on the page, depending on the connection status with the Bayeux server.

This is a common pattern using the /meta/connect and /meta/disconnect channels:

var _connected = false;

cometd.addListener('/meta/connect', function(message)
{
    // if (cometd.getStatus() === 'disconnecting' || cometd.getStatus() === 'disconnected')
    if (cometd.isDisconnected()) // Available since 1.1.2
    {
        return;
    }
    var wasConnected = _connected;
    _connected = message.successful;
    if (!wasConnected && _connected)
    {
        // Reconnected
    }
    else if (wasConnected && !_connected)
    {
        // Disconnected
    }
});

cometd.addListener('/meta/disconnect', function(message)
{
    if (message.successful)
    {
        _connected = false;
    }
}

One small caveat with the /meta/connect channel is that /meta/connect is also used for polling the server.
Therefore, if a disconnect is issued during an active poll, the active poll is returned by the server and this triggers the /meta/connect listener.
The initial check on the status verifies that is not the case before executing the connection logic.

Another interesting usage of meta channels is when there is an authentication step during the handshake.
In this case the registration to the /meta/handshake channel can give details about, for example, authentication failures.

CometD JavaScript Publish

JavaScript CometD API: Publishing

The publish() method allow you to publish data onto a certain channel:

cometd.publish('/mychannel', { mydata: { foo: 'bar' } });

You cannot (and it makes no sense) to publish to a meta channel, and you can publish to a channel even if you are not subscribed to that channel.
However, you have to handshake before being able to publish.

As with other JavaScript CometD API, publish() involves a communication with the server and it is asynchronous: it returns immediately, well before the Bayeux server has received the message.

Note
Calling publish() does not mean that you have published the message when publish() returns.

If you have to publish several messages to different channels, you may want to use message batching.

CometD JavaScript Disconnection

JavaScript CometD API: Disconnecting

The JavaScript CometD implementation performs automatic reconnect in case of network or Bayeux server failures.
The reconnect parameters are described in the configuration section.

Short Network Failures

In case of temporary network failures, the client is notified through the /meta/connect channel (see this section about meta channels) with messages that have the successful field set to false (see also the archetypes in the primer as an example).
However, the Bayeux server may be able to keep the client's state, and when the network resumes the Bayeux server may behave as if nothing happened.
The client in this case just re-establishes the long poll, but any message published by the client during the network failure is not automatically re-sent (though it is possible to be notified, through the /meta/publish channel, of the failed publishes).

Long Network Failures or Server Failures

If the network failure is long enough, the Bayeux server times out the lost client, and deletes the state associated with it. The same happens when the Bayeux server crashes (except of course that the state of all clients is lost).
In this case, the reconnection mechanism on the client performs the following steps:

  • a long poll is re-attempted, but the server rejects it with a 402::Unknown client error message
  • a handshake is attempted, and the server normally accepts it and allocates a new client
  • upon the successful re-handshake, a long poll is re-established

If you register with meta channels, be aware of these steps, since a reconnection may involve more than one message exchange with the server.

Disconnecting

Calling the JavaScript CometD API disconnect() result in a message being sent to the Bayeux server, so that it can cleanup any state associated with that client.
As with all methods that involve a communication with the Bayeux server, it is an asynchronous method: it returns immediately, well before the Bayeux server has received the disconnect request.
If the server cannot be reached (because it is down or because of network failures), the JavaScript CometD implementation will stop any reconnection attempt and cleanup any local state.
It is normally safe to ignore if the disconnect() call has been successful or not: the client is in any case disconnected, its local state cleaned up, and if the server has not been reached it will eventually time out the client and cleanup any server-side state for that client.

Tip
If you are debugging your application with Firebug, and you shutdown the server, you'll see in the Firebug console the attempts to reconnect.
To stop those attempts, simply type in the Firebug command line: dojox.cometd.disconnect() (for Dojo) or $.cometd.disconnect() (for jQuery).

CometD JavaScript Message Batching

JavaScript CometD API: Message Batching

It is often needed by an application to send several messages to possibly different channels.

One, naive, way of doing it is the following:

cometd.handshake();

// Warning: non-optimal code
cometd.publish('/channel1', { product: 'foo' });
cometd.publish('/channel2', { notificationType: 'all' });
cometd.publish('/channel3', { update: false });

You may think that the 3 publishes will leave the client one after the other, but that's actually not the case.
Remember that publish() is asynchronous (so it returns immediately), so the 3 publish() calls in sequence may return well before a single byte hits the network.

What happens is that the first publish() will be executed immediately, and the other 2 will be put in a queue, waiting for the first publish() to complete.
A publish() is complete when the server received it, the server sent back the meta response, and the client received the meta response for that publish.
When the first publish is completed, the second publish is executed and waited to complete. After that, finally the third publish() is executed.

Since CometD 1.1.2, a new configuration parameter called autoBatch can be set to true, allowing the implementation to automatically batch messages that have been queued up.
So, in the example above, the first publish() will be executed immediately, and when it completes, the implementation will batch the second and third publish() into one request to the server.
The autoBatch feature may be interesting for those systems where events received asynchronously and unpredictably, either at a fast rate or in bursts, end up in generating a publish() to the server: in such cases, using the batching API is not effective (as each event would generate only one publish()).
A bursts of events on the client will generate a bursts of publish() to the server, but this mechanism batches them automatically, making the communication more efficient.

This queueing mechanism is needed to avoid to queue a publish() behind a long poll. If not for this mechanism, the browser would receive 3 publish requests but only has 2 connections available, and one is already occupied by the long poll request. So the browser may decide to round robin the publish requests, so that the first publish goes on the second connection (remember that the first connection is already busy with the long poll request), which is free and it is actually sent over the network, schedule the second publish to the first connection (after the long poll returns), and schedule the third publish again to the second connection, after the first publish returns.
The result is that if you have a long poll timeout of 5 minutes, the second publish request may arrive to the server 5 minutes later than the first and the third publish request.

You can optimize the 3 publish using batching, which is a way to group messages together so that a single Bayeux message actually carries the 3 publish messages.

cometd.handshake();

cometd.batch(function()
{
    cometd.publish('/channel1', { product: 'foo' });
    cometd.publish('/channel2', { notificationType: 'all' });
    cometd.publish('/channel3', { update: false });
});

// Alternatively, but not recommended
cometd.startBatch()
cometd.publish('/channel1', { product: 'foo' });
cometd.publish('/channel2', { notificationType: 'all' });
cometd.publish('/channel3', { update: false });
cometd.endBatch()

Note how the 3 publish() calls are now within a function passed to batch().
Alternatively, but less recommended, you can surround the 3 publish() calls between startBatch() and endBatch().

Warning
Remember to call endBatch() after having called startBatch().
If you don't, for example because an exception is thrown in the middle of the batch, your messages will continue to queue up, and your application will not work as expected.

Function batch() already does the correct batching for you (also in case of errors), so it's the recommended way to do message batching.

When a batch is started, subsequent API calls are not sent to the server, but instead queued up, until the batch is ended.
The end of the batch packs up all the queued messages into one single Bayeux message and send it over the network to the Bayeux server.

Message batching allow an efficient utilization of the network, as instead of making 3 requests/responses cycles, batching makes only one request/response cycle.

Batches can be made up of different API calls:

var _subscription;

cometd.batch(function()
{
    cometd.unsubscribe(_subscription);
    _subscription = cometd.subscribe('/foo', function(message) { ... });
    cometd.publish('/bar', { ... });
});

Batched messages will be processed by the Bayeux server in the order they are sent.

If you still want to risk and use the startBatch() and endBatch() calls, remember that they must be done from the same context of execution; message batching has not been designed to span multiple user interactions.
So, for example, it would be wrong to start a batch in, say, functionA (triggered by user interaction), and ending the batch in functionB (also triggered by user interaction and not called by functionA).
Similarly, it would be wrong to start a batch in functionA and then schedule (using setTimeout()) the execution of functionB to end the batch.

CometD JavaScript Transports

JavaScript CometD Transports

The Bayeux specification defines two mandatory transports:

  • long-polling
  • callback-polling

The JavaScript CometD implementation implements exactly these two transports.

For most recent browsers (such as Firefox 3.5) it is possible to use the long-polling transport also for cross-domain Bayeux communication, see below the cross-domain mode.

The long-polling Transport

The long-polling transport is the default transport.
This transport is used when the communication with the Bayeux server happens on the same domain, and in the cross-domain mode (see below).
The data is sent to the server by means of a POST request with Content-Type text/json via a plain XMLHttpRequest call.

The callback-polling Transport

The callback-polling transport is the transport that is used when the communication with the Bayeux server happens on a different domain (when the cross-domain mode is not supported, see below for the cross-domain mode section).

It is well known that XMLHttpRequest calls have restrictions when the invocation is directed to a domain different from the one the script has been downloaded (but see below the cross-domain mode for an alternative solution).
To overcome XMLHttpRequest restrictions, this transport uses the JSONP script injection: instead of using XMLHttpRequest it injects a <script> element whose src attribute points to the Bayeux server.
The browser will notice the script element injection and performs a GET request to the specified source URL.
The Bayeux server is aware that this is a JSONP request and replies with a JavaScript function that is then executed by the browser (and that calls back into the JavaScript Cometd implementation).

There are three main drawbacks in using this transport:

  • The transport is chattier.
    This is due to the fact that the browser executes the injected scripts sequentially, and until a script has been completely "downloaded", it cannot be executed.
    For example, imagine a communication that involves a script injection for the long poll, and a script injection for a message publish. The browser injects the long poll script, a request is made to the Bayeux server, but the Bayeux server holds the request waiting for server-side events (so the "script" is not "downloaded"). Then the browser injects the publish script, the request is made to the Bayeux server, which replies (so the "script is "downloaded"). However, the browser does not execute the second script, because it has not executed the first yet (since its "download" is not finished). In these conditions, the publish would be executed only after the long poll returns. To avoid this situation the Bayeux server, in case of callback-polling transport, resumes the client's long poll for every message that arrives from that client, and that's why the transport is chattier: the long poll returns more often.
  • The message size is limited.
    This is necessary to support IE7, that has a 2083 character limit for GET requests.
  • The reaction to failures is slower.
    This is due to the fact that if the script injection points to a URL that returns an error (for example the Bayeux server is down), this is silently ignored by the browser.

The cross-domain Mode

Firefox 3.5 introduced the capability for XMLHttpRequest calls to be performed towards a different domain (see here).

As of version 1.0.0.rc0, this is supported also in the JavaScript CometD implementation, with no configuration necessary on the client (if the browser supports XMLHttpRequest cross-domain calls, they will be used) and with a bit of configuration for the server. Refer to this document for the server configuration.

To use the cross-domain mode, you need:

  • a cross-domain compliant browser (for example Firefox 3.5)
  • a compliant server (for example Jetty configured with the CrossOriginFilter)

With this setup, even when the communication with the Bayeux server is cross-domain, the long-polling transport will be used, avoiding the drawbacks of the callback-polling transport.